Robotic tattooing systems and related technologies

ABSTRACT

An automatic tattoo apparatus can be used to robotically apply tattoos. A customer can shop on an online tattoo marketplace to select designs created by various artists located anywhere. The online tattoo marketplace can process and manage payments, artist and/or customer profiles, bookings, tattoo design uploads, browsing and design selection, design changes, and/or perform other actions. The automatic tattoo device can apply tattoos precisely, quickly, and may with reduced pain. The tattoo apparatus can apply a wide range of different types of tattoos, including but not limited to micro tattoos, dotwork, blackwork tattoos, realism tattoos, fine-line tattoos, etc.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 17/157,935, filed Jan. 25, 2021, which is acontinuation-in-part of International Patent Application No.PCT/US2020/043588, filed Jul. 24, 2020, which claims priority to and thebenefit of U.S. Provisional Application No. 62/878,673, filed Jul. 25,2019, and U.S. Provisional Application No. 62/964,579 filed Jan. 22,2020, all of which are hereby incorporated by reference in theirentireties for all purposes.

TECHNICAL FIELD

The present disclosure relates generally to systems, devices, andmethods for tattooing and applying substances to skin.

BACKGROUND

To apply a tattoo, a tattooing device is held by a tattoo artist whilethe tattooing device vibrates a needle to inject pigment into the skin.If the injection is too deep, it may have a different hue due toscattering or may look blurred due to subdermal diffusion. If it is toosuperficial, it may not be held in proper position and may migrate toproduce a blurred image or be gradually removed to produce a faded imageas the dermis is recycled. Unfortunately, artistic ability variesbetween tattoo artists, and a particular tattoo artist may be unable toapply visually appealing tattoos. Tattoo artists may develop anexpertise applying particular types of tattoos, such as micro tattoos,dotwork, blackwork tattoos, realism tattoos, or fine-line tattoos. Anindividual may want a tattoo that cannot be produced by a local tattooartist, so the individual may travel to visit tattoo artists at otherlocations. In-demand tattoo artists often have exceptional skill thatcannot be adequately replicated by other tattoo artists, so they mayrequire booking weeks, months, or years in advance and the tattoos canbe expensive. Accordingly, conventional tattooing equipment andtechniques have numerous drawbacks.

SUMMARY

In some embodiments, an automatic tattoo apparatus can be used torobotically apply tattoos. A customer can shop on an online tattoomarketplace to select designs created by various artists locatedanywhere. The online tattoo marketplace can manage payments, artistand/or customer profiles, bookings, tattoo design uploads, browsing anddesign selection, design changes, and/or perform other actions. Theautomatic tattoo device can apply tattoos quickly with reduced pain. Thetattoo apparatus can apply a wide range of different types of tattoos,including but not limited to micro tattoos, dotwork, blackwork tattoos,realism tattoos, fine-line tattoos, etc.

The online tattoo marketplace can provide an augmented reality shoppingexperience enabling the customer to see how the tattoo will look at aparticular target site on the body. Once a tattoo is selected, theonline tattoo marketplace can notify a designated retail location of thepurchase. The tattoo marketplace can supply the designated retaillocation with a token (e.g., a digital token, credit, etc.) or licenseto apply a tattoo design. An artist can receive payment for theapplication of the tattoo. The online tattoo marketplace can be used toprovide graphics and designs from tattoo artists, non-tattoo artistssuch as visual artists, artistic celebrities, influencers, brands,artwork provided by customers themselves, or other sources. This allowscustomers to access artwork irrespective of an artist's physicallocation. In some embodiments, the artist can receive payment based onroyalties, commissions, or other payment schemes. The online tattoomarketplace can include original designs, limited edition designs,resident designs, custom lettering, custom designs, customer provideddesigns, or other designs. Additionally, the online tattoo marketplacemay offer other goods and services including but not limited to tattooauctions or sale of artwork (e.g., original works and/or prints). Afterthe tattoo apparatus has applied the art, one or more pictures can besupplied to the marketplace, tagged to the artists/studio. The picturescan either be taken by the tattoo machine or by a mobile phone, tablet,or other image capture device of the tattoo recipient, artist, etc.

The automatic tattoo apparatus can communicate with an artist ororiginator of the graphic and/or design via a network (e.g., a wide areanetwork). A remote server can store the designs/graphics available viathe online tattoo marketplace so the tattooing can be performed at anylocation (e.g., tattoo studio or retail location). For example,tattooing can be performed at retail locations with one or moreautomatic tattoo apparatuses that can be local and convenient for thecustomer. Each retail location may additionally provide other goods orservices to offer an elevated tattooing experience, such as after-tattoolotion and sunscreen, bandages, merchandise, etc. An operator of theautomatic tattoo apparatus may require less and/or different expertisethan a traditional tattoo artist. The automatic tattoo apparatus canapply tattoos visually the same as the originator of the graphic and/ordesign who can be an in-demand tattoo artists with exceptional skill.Individuals can obtain tattoos of artwork from an artist without havingto travel to or book a session with that artist. This allows artwork tobe reproduced at a wide range of locations. In some embodiments, theautomatic tattoo apparatus is in the form of a tattoo robot capable ofapplying tattoos. The tattoo robot can include one or more controllers,robotic arms, tattoo needle assemblies, etc.

At least some embodiments can include a tattoo apparatus comprising atattoo shuttle configured to carry a tattoo needle, at least one sensorconfigured to measure at least one characteristic of a portion of acustomer's skin, a machine vision device positioned to obtain one ormore images of the portion of skin, and at least one controller. Thecontroller can be configured to calculate a skin position and/or a skindeformation based on the obtained images and/or sensor signals, andcontrol a puncture depth based at least in part on at least one of theskin position, the skin deformation, or the characteristic(s) of theportion of skin. The needle can be disposable. In some embodiments, thetattoo apparatus can apply tattoos to articles made of naturalmaterials, synthetic materials, or combinations thereof. For example,the tattoos can be applied to belts, clothing, wallets, etc. In someembodiments, the tattoo apparatus is a portable handheld tattooingapparatus with integrated components.

In at least some embodiments, the sensor can be configured to measureinformation about a puncture operation on the portion of skin. Theinformation can include a load applied to the needle structure duringpuncture, an acceleration of the needle structure, a speed of the needlestructure, a velocity of the needle structure, an angular position ofthe needle structure, an impedance of contact between the needlestructure and skin while in contact with the portion of skin relative toan impedance of the portion of skin alone, and/or an amount ofvibration. Accelerometers, gyroscopes, and position sensors can measurethe acceleration, rotation, and/or position of the needle or othercomponents. The characteristic(s) of the portion of skin can alsoinclude, but is not limited to, a skin elasticity, impedance, and/orthickness (e.g., thickness of the skin, thickness of each layer of theskin, etc.). In some implementations, a force/pressure sensor is used todetermine the load and based on displacement output, skin elasticity isdetermined. Electrical sensors can monitor tissue impedance and based onchanges on measured impedance during punctures. Changes in impedance canbe correlated to when the needle passes through tissue, at a particulardepth, etc. Identification of tissue layers, thicknesses of tissuelayers, and other tissue information can be determined based on theoutput from such sensors.

The controller can be configured to determine a depth of a tissue layerinterface based at least in part on a relationship between a forceapplied to the portion of skin and the depth of the tissue layerinterface. Determination of the skin deformation can be based at leastin part on an initial deflection of the portion of skin resulting from aforce applied to the portion of skin. The controller can be configuredto determine characteristics of leather and other materials, whethernatural or synthetic.

At least some embodiments can include a tattoo device comprising atleast one sensor configured to detect skin puncture events for variousneedles, at least one characteristic of a portion of skin, or the like.The characteristic can include a depth of a tissue layer interface, anda means for controlling a piercing depth (i.e., puncture depth) based atleast in part on the measured characteristic of the portion of skin. Insome embodiments, the skin puncture event can relate to one or more of(1) the initial contact with the skin, (2) the initial epidermisfailure, (3) where the needle is at the junction of an epidermal and adermal layer of the skin, (4) where the needle tip is at the junctionbetween a dermal and a hypodermal layer of the skin, (5) where theneedle is at its deepest position and/or (6) where the needle exits theskin. The characteristic of the skin can include at least one of a skinelasticity, impedance, or thickness. The characteristics of undeformedor deformed skin can be determined. An optical sensor can be used todetermine one or more characteristics of undeformed skin whereas agalvanic sensor or an electrical sensor can be used to determine one ormore characteristics of deformed skin. Characteristics of the skin indifferent states (e.g., deformed state, undeformed state, etc.) can beused to monitor a site before, during, and after application of thetattoo.

In some embodiments, the sensor can be configured to detect the depth ofthe needle when a puncture event occurs based, at least in part, on arelationship between a force applied to the portion of skin and thedepth of the tissue layer interface. The sensor can be furtherconfigured to detect a deflection distance or initial deflection of theskin resulting from a force applied to the portion of skin. In someembodiments, the sensor measures the deflection of the skin caused by aneedle.

In some embodiments, the sensor can detect the position and/or depth ofthe needle, when a puncture event occurs, based at least in part onimpedance. For example, the sensor can detect the variation of a contactconductivity of the needle against the skin relative to the conductivityof the skin alone. The sensor can be further configured to detect adeflection distance (e.g., initial deflection to puncture) of the skinresulting from a force applied to the portion of skin. In someembodiments, the sensor measures, whether directly or indirectly, thedeflection of the skin caused by a needle.

At least some embodiments can include a method for robotic tattooingthat includes measuring at least one characteristic of a portion of skincorresponding to a dot position and controlling a piercing depth (i.e.,puncture depth) for the dot position based at least in part on thecharacteristic(s) of the portion of skin. In some embodiments, the dotposition can be one of a plurality of dot positions, and the method canbe repeated for each dot position. In some embodiments, the dot positioncan be one of a plurality of dot positions, and the measuring step canbe repeated for some of the dot positions. The characteristic of aportion of skin for the remainder of the plurality of dot positions canbe determined based at least on interpolation from the measuredcharacteristic of the portion of skin of the measured dot positions. Themeasured characteristic of the portion of skin can include at least oneof a skin elasticity, impedance, or thickness. In some embodiments, themeasured characteristic can include one or more mechanicalcharacteristics (e.g., elasticity of tissue, puncture strength, and/ortear strength), electrical characteristics (e.g., impedance), dimensions(e.g., position and/or layer thickness), or the like.

In at least some embodiments, the method can further comprise detectinga position and/or depth of the needle when a puncture event occurs basedat least in part on a relationship between a force applied to theportion of skin and the depth of the tissue layer interface. In at leastsome other embodiments, the method may further comprise detecting a skindeformation of the portion of skin based at least in part on an initialdeflection of the portion of skin resulting from a force applied to theportion of skin. The skin can be leather, skin of a living animal (e.g.,a human, livestock, etc.), or the like. The method can also be used toapply tattoos to synthetic materials.

The method, in at least some embodiments, can further comprise detectinga position and/or depth of the needle when a puncture event occurs basedat least in part on the variation of the contact conductivity of theneedle against the skin relative to the conductivity of the skin alone.In some embodiments, the method may further comprise detecting a skindeformation of the portion of skin based at least in part on an initialdeflection distance of the portion of skin resulting from a forceapplied to the portion of skin.

At least some embodiments can include a method for tattooing skin,comprising: acquiring at least one skin puncture property, updating adot parameter table with a machine setting based on the acquired atleast one skin puncture property, and controlling deposit of a substanceinto the skin based at least in part on the updated dot parameter table.In some embodiments, the method may further comprise applying a stencil(e.g., a stencil with a plurality of dot positions) prior to acquiringthe skin puncture property. In some embodiments, the skin punctureproperty includes a skin deformation measurement. The skin deformationmeasurement can be based at least in part on an initial deflection ofthe skin resulting from a force applied to the skin.

The dot parameter table can include a plurality of dot positions and theupdating can further include updating a machine setting for each of theplurality of dot positions. In some embodiments, updating the dotparameter table step further comprises determining the machine settingbased on interpolation from the skin puncture property. The skinpuncture property can include a skin elasticity, impedance, tissue layerdepth, tissue layer thickness, and/or layer junction locations.

In at least some embodiments, the method can further comprise detectinga position and/or depth of the needle when a puncture event occurs basedat least in part on a relationship between a force applied to theportion of skin and the depth of the tissue layer interface. The depositof a substance can be controlled based at least in part on the positionand/or depth of the needle when the puncture event occurs.

In at least some embodiments the method can further comprise detecting aposition and/or depth of the needle when a puncture event occurs basedat least in part on the variation of the contact conductance, relativeto the conductivity of the skin alone. The deposit of a substance in theskin can be controlled based at least in part on the depth of the needlewhen a puncture event occurs. In some embodiments, the contactconductivity can be monitored to evaluate changes of the contactconductivity as the puncture depths increases, position of the needlechanges, or the like.

At least some embodiments can include methods for managing amarketplace. The method can include providing a user interfaceillustrating one or more tattoo designs. The method can includereceiving a selection of one or more designs and providing the one ormore designs to an automatic tattooing apparatus at a retail location inassociation with a digital token. The automatic tattooing apparatusapplies the one or more designs to a tattoo recipient in response toreceiving an indication of the digital token. A user can select andreceive a tattoo by browsing an online tattoo marketplace, selecting oneor more designs, going to a retail location housing an automatictattooing apparatus, and receiving a tattoo of the one or more selecteddesigns from the automatic tattooing apparatus. The browsing andselecting steps can be performed by different users (e.g., a customer,an artist, or the like) through a mobile application, computer, website,or the like. In other embodiments, the browsing and selecting steps canbe performed by a user through a computing device, such as a smartphone,an augmented reality device, a computer. In some embodiments, designsavailable on the online tattoo marketplace could have been contributedby one or more of an artist, another user, company, or the user. A usercan select a tattoo selection tool via an online tattoo marketplace, atthe retail location, or any other suitable location. In someembodiments, the user can use the computing device to preview thelocation and the design of the selected tattoo. For example, the usercan use augmented reality to view the location of the tattoo on his orher body. The user can then accept the location or reposition thattattoo. The tattoo system can lock the tattoo location based on theuser's acceptance.

The tattoo design can be rendered on an image of a portion of the skinor on the skin itself. In some embodiments, a light projecting devicecan render a tattoo design on the skin. In some embodiments, an image isvisualized on the skin using a real time feed from an augmented realityapparatus, such as a smart phone, a smart TV, AR googles, a smartmirror, a computer, or any user device containing a camera for real timefeed and a screen or lenses for visualization of augmented realityimages. A method for altering the positioning and scaling of a tattoodesign rendering can be based on user input.

An artist who created at least one of the selected designs can receivepayment for each created design that was received as a tattoo. In someembodiments, the retail location can be a location remote from an artistwho created the design. The artist can receive information aboutalterations to tattoo designs to help the artist generate new tattoodesigns.

At least some embodiments are directed to an automated apparatusconfigured to analyze a site and to puncture a subject's skin at thesite. The analysis can include, without limitation, one or more opticalanalyses, electrical analyses, mechanical analyses, chemical analyses,or combinations thereof. The apparatus can puncture the subject skin toapply one or more liquids, medications, substances, or combinationsthereof. In some embodiments, the apparatus can perform multipleanalyses to perform a task, such as applying one or more tattoos. Theapparatus can perform analyses to position a piercing element (e.g., aneedle, needle array, etc.) for injecting a fluid (e.g., liquid ink,pigment, dyes, etc.) into one or more layers of skin or other tissue.The apparatus can be used in tattooing applications, medicalapplications, aesthetic applications, or other suitable uses.

In tattooing applications, a temporary pigment can be injected into theepidermal layer to provide a reference feature (e.g., a temporary dot).A permanent pigment can be injected into the dermal or other layer usingthe reference feature. The subject's body can naturally cause thereference feature to break down and be absorbed into the subject's bodyleaving only the permanent features (e.g., tattoo dots). An opticalanalysis can include using machine vision or computer vision to identifyreference features (e.g., applied and/or natural fiducials), landmarks,stenciling, applied dots (e.g., previously applied dots when applyingthe tattoo), skin features (e.g., moles, scars, hair, etc.), or thelike. The apparatus can be programmed to identify such features anddetermine one or more of the following: position of stenciling, tattooplacement, puncture sites (e.g., interrogation sites), volume of ink tobe applied at each puncture site, depth of puncture site, needlecharacteristics, and/or position information. If the subject's body partmoves during a session, the apparatus can identify the movement anddetermine an appropriate protocol for continuing one or more tasks basedon the new body position. This allows a tattoo to be robotically appliedwithout disrupting the session. The apparatus can apply a wide range ofsubstances, including fluids, gels, or other suitable substances. Forexample, during or after the session, the apparatus can inject one ormore medicants, analgesics, pigment enhancing agents, antibacterialagents, or other substances in the dermal and/or epidermal layers to,for example, reduce discomfort, promote healing, inhibit infection, orcombinations thereof. In a following session (e.g., a session days,weeks, or months after a tattoo is applied), the automated apparatus cananalyze the tattoo and identify areas to be modified by, for example,reapplying dots, touching up dots, etc. Images of the applied tattoo canbe compared to a virtual tattoo to identify the areas to be modified.

In non-tattoo applications, the apparatus can apply botulinum toxin(e.g., Botox®), anti-wrinkle agents, denervating agents, anti-acneagents, collagen, or the like. The apparatus can optically analyze asite and identify wrinkles (using a trained computer vision systemsimilar to that described below). Targeted wrinkles can be located alongthe subject face (e.g., along the forehead, surrounding the eyes, etc.)or any other location. The apparatus can determine one or more puncturesites based on characteristics (e.g., size, depth, location, etc.) ofthe wrinkles. The apparatus can inject one or more anti-wrinkle agentsat puncture sites to reduce or limit the appearance of the targetedwrinkles.

The apparatus can include one or more machine vision systems configuredfor imaging-based automatic inspection and analysis. The machine visionsystems can be configured for one-dimensional, two-dimensional, orthree-dimensional analysis and can include one or more image capturedevices, such as digital cameras. The machine vision system can analyzenon-planar surfaces, planar or flat surfaces, optically identifiablefeatures, and other features. The non-planar surfaces can include,without limitation, curved surfaces (e.g., highly-contoured regions ofskin), undulating surfaces, or the like. The flat surfaces can begenerally flat areas of tissue. A frame can be pressed against thesubject's tissue to flatten the site. The optically identifiablefeatures can include, without limitation, dots, tattoos (e.g., portionsof tattoos, entire tattoos, etc.), stenciling (e.g., dots, patterns ofdots, etc.), body parts, or the like. In some embodiments, the machinevision systems can be configured to perform, without limitation, one ormore line scans, area scans, triangulation data collection (e.g., 3Dimages suitable for triangulation), etc. The machine vision systems cancapture images and generate one or more maps based on the capturedimages. For example, captured images can be combined to generatemulti-dimensional maps (e.g., two-dimensional maps, three-dimensionalmaps, etc.), or the like. In some implementations, such a computervision system can use a machine learning model or other suitableanalytical models trained to identify desired features (e.g., skinlandmarks, stenciling, applied tattoo dots, wrinkles, moles, scars,hair, etc.). For example, a neural network can be trained to identifysuch features with supervised learning, applying training itemscomprising images with parts tagged as having or not having thesefeatures. The training data can be based on human tagged images, medialdatabases, etc. In various implementations, different types of neuralnetworks (e.g., deep neural networks, convolutional neural networks,etc.) can be used or other types of machine learning models (e.g.,decision trees, support vector machines, etc.) can be used. Further,different types of training can be applied (e.g., supervised,unsupervised, applying different types of loss or activation functions,etc.).

In some embodiments, the apparatus can include one or more cameras,sensors (e.g., 2D or 3D sensors such as laser-displacement sensors,imaging sensors, calibration sensors, etc.) for outputting data forinspection, feature identification, surface topology, area evaluation,volume measurement, or the like. The output from the sensors can be usedto produce, without limitation, images (e.g., digital images), maps oftarget sites, height maps (e.g., height maps generated from displacementof reflected lasers), or the like.

In some embodiments, a system can be used to analyze one or moreinterrogation sites to determine at least in part how to apply a tattooor a portion thereof. The interrogation sites can be punctured todetermine skin characteristics, including number of skin layers within acertain depth, dimensions of skin layers, characteristics of skin layers(e.g., elasticity, puncture characteristics, etc.), or the like.Puncture sites can be the same as or different from the interrogationsites. Puncture sites suitable for receiving dye can be selected basedon, for example, target site characteristics, the tattoo design to beapplied, stencils, etc. During the tattooing process, a tattoo site canbe periodically or continuously analyzed to adaptively adjust theapplication of the tattoo to enhance application by, for example,compensating for, without limitation, skin stretch, appearances ofapplied dots, body part movement, or the like.

Establishing skin position may not be sufficient to execute an accuraterendition of the tattoo. This is because skin is rarely in its relaxed,non-stretched or undeformed state during tattooing. When a tattoo isapplied to deformed skin, the tattoo may look stretched or contractedwhen the skin is relaxed. This may result in a less accurate tattooapplication, because skin is constantly stretched or deformed based onthe position of the body part being tattooed. In order to perform anaccurate rendition (e.g., a non-distorted rendition of a referencetattoo design), the system can measure deformation of the skin and applya compensation of the positioning.

The system can periodically or constantly compensate for the positionand deformation of the portion of the skin to identify the appropriatelocation of the applied ink. In-plane skin deformation is a field ofdisplacement associated with the surface of the skin as the skinstretches, contracts, and/or rotates. The skin deformation is null whenthe skin is in its relaxed state. The skin can have a non-nulldeformation when force is applied to it (e.g., when an object pushesagainst the skin, skin is stretched by natural movement of the body,etc.). If deformation is not compensated for, the applied tattoo maylook deformed or distorted when the skin is relaxed. In someembodiments, skin deformation is compensated for by analyzing skinfiducials in different states (e.g., an undeformed state, deformedstate, etc.). This is done by, for example, (i) scanning/analyzing theskin in an undeformed state to identify fiducials in an undeformed stateusing machine vision and/or (ii) applying a stencil (e.g., containing aknown pattern of fiducials) on the skin while the skin is in anundeformed relaxed state of the skin and tracking the stencil withmachine vision during the tattoo session. Deformation is tracked as thedisplacement field from the undeformed state of the skin to the state ofthe skin in the configuration in which the tattoo is performed.

In general, compensating for skin changes during tattooing can beimportant because skin is not in its undeformed state. Trackingfiducials during the tattooing process alone may not account for thedeformation occurring between the relaxed state of the skin and thepotentially deformed state. Image analyses can analyze skin over aperiod of time to determine how to compensate for skin deformation. Insome embodiments, a machine vision method analyzes one or more imagescollected during tattooing and compares it to the undeformed, relaxedskin analysis. The skin deformation compensation can be performed byapplying the same deformation to the portion of the tattoo design beingapplied in order to have an undeformed result when the skin is at rest(e.g., undeformed or in a natural state).

The tattoo systems can include a frame used to grossly maintain orimmobilize a part of the body containing the portion of the skin to betattooed. The frame can be configured to securely hold the body part. Insome embodiments, the tattoo system includes a contactor which maintainsor immobilizes a portion of the skin to be tattooed. The contactor canstabilize the skin distance to a needle head. The contactor can inhibit,limit, or prevent run-off liquids. The contactor can also integrate amovement detection apparatus or a suction system, and can position,hold, and/or flatten the skin to be tattooed.

A tattoo system can include cleaning an area by, for example, removingexcess fluids using a suction system. Nozzle geometry, angle of attack,and suction nozzle position can be selected to improve suction of targetsubstances (e.g., liquids) only. The suction system can provide suctionnear to or at the edge of a contactor. In some embodiments, a cleaningsystem includes one or more suction systems, lubricant and contactorheads. The lubricant can serve as a barrier to protect against stainsdue to excess fluids. The lubricant can facilitate sliding of thecontactor along the skin. The contactor can be configured to inhibit orprevent runoff of excess fluids. The system can be configured todetermine when cleaning procedures should be performed.

A method of tattooing includes pausing a tattooing operation andevaluating a new position if a movement is detected. In someembodiments, machine vision is used to detect movement of skin. In someembodiments, an optical sensor on a portion of skin that is not beingtattooed detects movement. In some embodiments, one or more non-opticalsensors, such as accelerometer or vibration sensors, detect themovement. The number, position, and functionality of the sensors can beselected based on desired movement detection, skin position, needleposition/movement, etc. In some embodiments, a robotic tattooing systemcan detect movement of a tattoo site or body part. The detected movementcan be analyzed to determine whether to adjust the tattooing protocol.The tattooing protocol can be adjusted to compensate for the movement.For example, a frame of reference of the tattooing protocol can beadjusted to match the detected movement of the body part. Additionallyor alternatively, the detected movement can be movement of the needle,contactor, or another component of the robotic tattooing system.Different types of movement can be analyzed to control the tattooingprocess.

In some embodiments, a tattoo system comprises a gross positioningactuator and a camera or machine visions system. The gross positioningactuator and camera can be used to position a tattoo head in contactagainst a body part. The gross positioning actuator and camera can beused to localize and position the tattoo needle with respect to theportion of the skin to be tattooed. The tattoo system can perform aone-stage or multi-stage tattooing process.

At least some systems disclosed herein can use natural or artificialfeatures or patterns for positioning. At least some robotic systems canidentify natural or artificial patterns for skin deformationidentification. One or more pattern-detection algorithms can be used toidentify fiducials (applied or natural fiducials), patterns (whethernatural or artificial patterns), skin changes, or other features ofinterest based on output from one or more machine vision systems. Infurther embodiments, a tattooing system can include one or moredisposable or reusable ink containers. The tattooing system can includea pump or refilling system for replenishing ink by, for example,replacing or refilling the ink containers. The ink containers can berefilled when the ink is at a low level or at a rate commensurate to thenumber of punctures performed.

The robotic systems disclosed herein can generate a digitalrepresentation of a selected tattoo design. Puncture settings can beselected for individual puncture sites, a group of puncture sites, orthe entire tattoo. In some embodiments, puncture settings areindividually determined for each puncture site for producing a dot. Thisallows for precise control of the appearance of each dot. In certainembodiments, the number of punctures per site can be controlled by theuser by, for example, inputting maximum and minimum number of puncturesper site. Other user inputs can be used to define ranges for parametersdisclosed herein. During tattooing, the system can periodically orcontinuously analyze applied punctures to modify or select puncturesettings for dots to be applied. This allows for adaptive control of thetattooing process. For example, if the robotic system identifies anabnormal region of tissue, the robotic system can compensate forvariations in skin characteristics to produce dots having a targetappearance. Dots with desired appearances can be formed at sites withvarying tissue properties (e.g., sites with scar tissue, sites with athicker or thinner epidermal layer, etc.), visual characteristics, orthe like.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a network diagram of a tattooing environment and system inaccordance with an embodiment of the disclosure.

FIG. 1B is a schematic isometric view of a tattooing system inaccordance with an embodiment of the disclosure.

FIG. 2 is a schematic isometric view of a tattoo shuttle of thetattooing system of FIG. 1B in accordance with an embodiment of thedisclosure.

FIG. 3A is a block diagram of a tattooing process in accordance with anembodiment of the disclosure.

FIG. 3B is a block diagram of a skin puncture property acquisition stepof the tattooing process in accordance with the embodiment of FIG. 3A.

FIG. 3C is a block diagram of a tattoo application process step(tattooing process step) of the tattooing process in accordance with theembodiment of FIG. 3A.

FIG. 4 is a block diagram of a tattooing process in accordance with anembodiment of the disclosure.

FIGS. 5A and 5B are illustrative diagrams of a stencil generationprocess in accordance with an embodiment of the disclosure.

FIG. 6A illustrates an image captured by an image capture device inaccordance with an embodiment of the disclosure.

FIG. 6B illustrates an image analysis using one or more machine visionalgorithms in accordance with an embodiment of the disclosure.

FIG. 7 is a block diagram of machine vision process in accordance withan embodiment of the disclosure.

FIG. 8 is a schematic isometric view of a tattooing system in accordancewith an embodiment of the disclosure.

FIG. 9 is a block diagram of an embodiment of a tattoo selection andapplication process in accordance with an embodiment of the disclosure.

FIG. 10A is an isometric view of an embodiment of a needle cartridge inaccordance with an embodiment of the disclosure.

FIG. 10B is an exploded view of the needle cartridge of FIG. 10A inaccordance with an embodiment of the disclosure.

FIGS. 11A and 11B are illustrative diagrams of an example rejection andexample approval of a stencil position, respectively, in accordance withan embodiment of the disclosure.

FIGS. 11C and 11D are simplified illustrative diagrams of an examplerejected stencil and an example approved stencil, respectively, inaccordance with an embodiment of the disclosure.

FIG. 12A is an illustrative diagram of an example puncture event inaccordance with an embodiment of the disclosure.

FIG. 12B illustrates a conductivity curve as detected by a galvanicsensor and data curves in accordance with an embodiment of thedisclosure.

FIG. 13 is a block diagram of steps of a puncture setting predictionprocedure in accordance with an embodiment of the disclosure.

FIG. 14 is an isometric view of a handheld tattoo device in accordancewith an embodiment of the disclosure.

FIG. 15 illustrates a handheld automatic tattoo apparatus in accordancewith an embodiment of the disclosure.

FIG. 16 is an exploded side view of a handheld device in accordance withan embodiment of the disclosure.

FIG. 17 is a block diagram of an embodiment of the procedure forapplying a tattoo using a handheld automatic tattoo apparatus inaccordance with an embodiment of the disclosure.

FIGS. 18A-18C illustrate a stenciling process for manual tattooing inaccordance with an embodiment of the disclosure.

FIG. 19 is a block diagram of an embodiment of a digital tattoo filegeneration process optimized for automatic tattooing in accordance withan embodiment of the disclosure.

FIG. 20 is a schematic block diagram illustrating subcomponents of acontroller in accordance with an embodiment of the disclosure.

FIG. 21 is a network diagram of a tattooing environment and system inaccordance with an embodiment of the disclosure.

FIG. 22 is a block diagram of an embodiment of a tattooing protocol inaccordance with an embodiment of the disclosure.

FIG. 23A shows an example of a visual art received from an artist.

FIG. 23B shows a digital tattoo image or file generated based on thevisual art of FIG. 23A.

FIG. 24A shows a reference tattoo design in a target configuration whenskin is relaxed.

FIG. 24B shows the applied compensated and uncompensated designs whenskin is deformed.

FIG. 24C shows the resulting tattoo designs of FIG. 24B on relaxed skin.

FIG. 25 shows a series of four tattoo dots robotically applied to humanskin and sensor data collected when the tattoo dots were applied.

FIG. 26 shows a series of tattoo dots produced with varying puncturesettings to control the visual outcome of tattoo dots.

DETAILED DESCRIPTION Tattooing Environment and Systems

FIG. 1A is a network diagram of a tattooing environment and system 60 inaccordance with an embodiment of the disclosure. A client or subject 70(“subject 70”) and operator 72 can be located at a tattoo studio 74. Thetattoo studio 74 can include one or more automated tattooing systems 76configured to apply a tattoo based on artwork selected by the subject70. The operator 72 can prepare the tattoo site and operate theautomated tattooing system 76.

A tattoo assistance system 78 can include, without limitation, one ormore computing devices and/or system and can provide data used by theautomated tattooing systems 76. The tattoo assistance system 78 canperform one or more steps of a tattooing process, such as generatingcalibration protocols, determining skin puncture properties, generatingtables (e.g., dot parameter tables, puncture data, etc.), processingimages, generating stenciling, generating tattooing protocols, or thelike. The tattoo assistance system 78 can include, for example, one ormore servers, processors, and memory storing instructions executable bythe one or more processors to perform the methods described herein. Insome embodiments, the server implemented can be a distributed “cloud”computing system or facility across any suitable combination of hardwareand/or virtual computing resources. The tattoo assistance system 78 cancommunicate with the automated tattooing systems 76, network 82, andother systems through communication channels 80.

The network 82 can be in communication with artwork providers 84 andusers/clients 86. The artwork providers 84 can be artists that uploadtattoo artwork to an online tattoo marketplace. In one embodiment, theonline tattoo marketplace may be a global online tattoo marketplacewhere artwork providers 84 may upload, license, and/or sell theirdesigns irrespective of their physical location. Artists may be paid aroyalty based on selection and/or licensing of their designs by subjectsthrough the app and/or website. The tattoo assistance system 78 canprovide or support a user interface illustrating one or more tattoodesigns. The tattoo assistance system 78 can receive a selection of oneor more design and provide the one or more designs to an automatedtattooing system 76 with a digital token. The automated tattooingsystems can use one or more designs to a tattoo recipient (e.g., subject70 in FIG. 1A) in response to receiving an indication of the digitaltoken. The tattoo assistance system 78 can include features andfunctionality discussed in connection with FIGS. 20 and 21 and performone or more of the steps (e.g., all of the steps) of algorithmsdiscussed in connection with FIGS. 3A-7, 9, 12A-13, 17, and 19.

The subject 70 can purchase a tattoo from one of the artwork providers84 who may be located at a remote location. The subject 70 can obtainartwork for generating a high-quality tattoo that appears similar tooriginal artwork provided by the artist. The tattoo system 76 canreproduce artwork more consistently than a human tattoo artist.Accordingly, individuals across the world can purchase artwork form anartist and receive a tattoo of the artwork without requiring that theindividual travel to the artist. The tattoo system 76 can replicatetattoos from an in-demand tattoo artist without requiring booking withthat artist, thereby reducing the time to receive the tattoo and costs.Additionally, the tattoo system 76 can apply, for example, microtattoos, dotwork, blackwork tattoos, realism tattoos, and/or fine-linetattoos. Tattoos can be applied based on artwork from individualslocated throughout the world. The tattoo system 76 can include one ormore robotic arms (e.g., multi-axis arms, etc.), linear actuators,rails, motors, gantries, controllers, and other suitable components formanipulating and positioning needles to produce the tattoo.

The tattoo assistance system 78 can include at least one database 88 andmodule 89. The database 88 can be configured to store artwork,protocols, tattoo data, skin data, stencil data, client data, or thelike. The module 89 can be configured with one or more algorithms forperforming processes disclosed herein and discussed in connection withFIGS. 3A-7, 9, 11A-13, and 17-18. Some or all of the functionalitydescribed herein with respect to the module 89 may also be performed bythe tattoo systems/apparatuses, and vice-versa. The module 89 can alsoinclude some or all of the functionality and features described hereinwith respect to the controller of FIG. 20 or other controllers disclosedherein. A remote server of the module 89 can store the designs/graphicsavailable via the online tattoo marketplace so the tattooing can beperformed at any location (e.g., tattoo studio or retail location). Forexample, tattooing can be performed at retail locations with one or moreautomatic tattoo apparatuses that can be local and convenient for thecustomer. In some embodiments, the tattoo assistance system 78 cancommunicate with an artist or originator of the graphic and/or designvia the network 82 (e.g., a wide area network).

The subject 70 and users/clients 86 can use a user device to selectartwork, purchase tattoos, input preferences, submit payment, managecredits/tokens, or the like. Browsing and selection of artwork may bedone via a mobile app and/or website that allows access to the onlinetattoo marketplace, through which subjects may perform actionsincluding, but not limited to, browsing, selecting, saving, ratingdesigns, uploading, creating a profile, booking appointments,participating in auctions, or buying. In one embodiment, the onlinetattoo marketplace may be a global online tattoo marketplace whereartists or users may upload, license, and/or sell their designsirrespective of their physical location. Artists may be paid a royaltybased on selection and/or licensing of their designs by subjects throughthe app and/or website. Browsing, selection, and payment process mayvary by location, as well as by individual artists. Exemplary userdevices include, without limitation, a personal computer (PC), a laptop,a tablet computer, or a smartphone. Generally, the user device caninclude a display and/or one or more processors. The displays can offerthe user a visual interface for interaction with the system, asdiscussed in connection with FIG. 9 (e.g., user device 509 of FIG. 9).The tattoo studio 74 can have different robotic tattooing machinesdisclosed herein for producing a wide range of different types oftattoos to the subject 70.

FIG. 1B is a schematic isometric view of a tattooing system 90 inaccordance with an embodiment of the disclosure. The tattooing system 90is suitable for use in the environment of FIG. 1A and can include atattooing apparatus 100 and at least one controller 108. The tattooingsystem 90 can determine a tattoo protocol or receive a tattoo protocolbased on a target tattoo site on the subject. The tattooing apparatus100 can apply the selected tattoo based on the tattoo protocol. Duringthe tattoo session, the subject's body part can rest on a rest surface102.

The tattooing apparatus 100 can include a cantilevered tattoo machine101 (“tattoo machine 101”), a tattoo frame 103, and a tattoo shuttle 104configured to carry a tattoo needle. The tattoo machine 101 can move thetattoo shuttle 104 while the tattoo frame 103 is against the targettattoo site. The tattooing apparatus 100 can also include one or moresensors 116 and at least one controller 109. The sensors 116 can becarried by the shuttle 104 and/or a component of the shuttle 104 andconfigured to measure at least one characteristic of a subject's skin.The tattooing process can be controlled based at least in part on themeasured characteristic(s) of the portion of skin, such as skinelasticity, impedance, or thickness (including thicknesses of one ormore skin layers).

The cantilevered tattoo machine 101 can be a structural elementconnected to the tattoo shop floor, which holds the rest surface 102,the tattoo frame 103, and the tattoo shuttle 104. The cantileveredtattoo machine 101 can be configured to provide structural support andstability to the tattooing system 90 and its components. In someembodiments, the cantilevered tattoo machine 101 can include motors(e.g., drive motors, stepper motors, etc.), robotic arms (e.g.,multi-axis arms), gantry devices, linear slides, rails, sensors (e.g.,position sensors, accelerometers, etc.), motors, rails, or the like.

With continued reference to FIG. 1B, the tattooing apparatus 100 can beactuated through a cantilevered X gantry 105 along an axis, illustratedas an X axis. The X gantry 105 may be a mechanical gantry that moves onthe X axis and connects the tattoo shuttle 104 to the cantileveredtattoo machine 101. A Y axis may be orthogonal to the X axis in a planeof the tattoo frame 103. An N axis may be normal to a plane formed bythe X and Y axes. A Z axis may be formed with a degree of inclinationrelative to the N axis. In one embodiment, the Z axis is not orthogonalto the plane formed by the X and Y axes. For example, the Z axis mayhave a 10 degree, 15 degree, or 20 degree inclination to the XY normal(N axis) in the XZ plane, and 0-degrees in the YZ plane. In anotherexample, the Z axis may have more or less than a 15-degree inclinationin the XZ plane, and more or less than about 0-degrees in the YZ plane.In another embodiment, the Z axis is orthogonal to the plane formed bythe X and Y axes.

The rest surface 102 can be a pad with a set of inflatable bladders orpneumatic actuator for precise alignment of a body part to be tattooed.For example, the pneumatic actuator may be configured to maintain atattoo area of the body in contact with the tattoo frame 103, whileapplying low enough pressure as to not interrupt blood perfusion of thebody part. The rest surface 102 may be configured to orient the bodypart to be tattooed in a relaxed position for the subject. The pad ofthe rest surface 102 may vary in size, shape, and configuration. Forexample, the pad may be larger or smaller than the tattoo frame 103. Ina particular embodiment, the pad may be larger than the largest tattooframe 103. In another example, the pad may comprise of multiple segmentsand/or shapes. The rest surface 102 can be raised or lowered so that itsheight is capable of being set manually, or automatically, with ndegrees of freedom. For example, rest surface height can be set with 3+1degrees of freedom. In another embodiment rest surface height can havemore or less than 3+1 degrees of freedom. The rest surface 102 can berotated manually and/or set monobloc with the cantilevered tattoomachine 101. In some embodiments, the pad 102 may not be actuated ornecessary and the tattoo machine 100 may be positioned to apply theappropriate contact force to the body part regardless of its orientationand rest surface, such that the contactor 120 of FIG. 2 is in contactwith the area to be tattooed and/or for the frame 103 to hold the bodypart of interest in place.

The tattoo frame 103 may be a flat frame surface in contact with theskin to isolate the area where the tattoo is to be applied. The tattooframe 103 may be fixed in the YN direction and all rotations withrespect to the tattoo shuttle 104, and/or also fixed with respect to thecantilevered tattoo machine 101. The tattoo frame 103 may comprise avariety of shapes and sizes. In one embodiment, the tattoo frame 103 canbe generally rectangular with a window (e.g., a polygonal window,rectangular window, etc.) to isolate the area of skin where the tattoois to be applied. In other embodiments, the tattoo frame may be, forexample, circular, ovular, or any other shape. The tattoo frame 103 maybe equal to, larger, or smaller than the rest surface 102. The size andshape of the tattoo frame 103 can generally correspond to an area of thebody to be tattooed. For example, in one embodiment, an appropriatetattoo frame 103 may generally match the size of the area of the body tobe tattooed. In some embodiments, the tattoo frame 103 size may bebetween about 20×20 mm to about 100×300 mm. Additionally, the tattooingapparatus 100 may have a plurality of tattoo frames 103 that maintaincontact with the skin and expose a tattoo zone. In some embodiments,tattooing apparatus 100 may utilize one or more tattoo frames 103 thatare interchangeable and chosen based on the desired tattoo. The tattoozone can be rotated and repositioned to match a location of the tattoo.In some embodiments, the tattoo frame 103 can be omitted and other meanscan be employed to hold the body part of interest in place and/or atleast detect movement of the body part of interest before, during,and/or after the tattooing process.

The contactor 102 and/or tattoo frame 103 (or other component of thetattoo apparatus 100) disclosed herein can help maintain or immobilizethe body part while the contactor 102 maintains a desired distance(e.g., a constant distance, a distance within a range, distance from theskin to the tattoo needle, etc.). If the client twitches or moves thebody part being tattooed, the tattoo frame 103 can provide enoughresistance to limit or avoid accidental gross movement. On the otherhand, the contactor force applied to the skin allows for compensation ofsmall movement of the skin in the vicinity of the contactor 102. Forexample, if the body part moves by a small amount, and because of theskin friction against the contactor 102 and the inherent elasticity ofskin, the portion of skin in the window of the contactor can remaingenerally static with respect to the contactor. The contactor role inthis example is for maintaining the skin in place. Another role of thecontactor 102 can be to maintain skin height position. The skin canremain in contact with the contactor window edge during tattooing. As aresult, the distance from the needle to the skin can be known within aprecision equal to or lower than 0.1 millimeter, 0.3 millimeters, 0.5millimeters, 1 millimeters, or 2 millimeters, or other suitabledistances. The skin may only deform a small amount within the window andcan be easily compensated for by varying the needle extension and byusing, for example, a galvanic sensing system, one or more positionsensors, and/or one or more range-finding sensors. Additionally oralternatively, the contactor 102 can apply a shear force to the skinwhen moving from position to position, which may stretch the skin. Thestretched skin can have more uniform properties than relaxed skin. Forexample, when the contactor 102 slides across the skin, the skin may bestretched due to the friction between the contactor 102 and the skin.The system can include one or more sensors capable of detecting appliedforces (shear forces, compressive forces, etc.), skin stretching, skinmovement, etc.

The contactor 102 can serve as a barrier against the skin to preventrunoff of liquids, such as lubricant, bodily fluids, and injectedsubstances, such as ink. For example, during a puncture, excess ink mayaccumulate in the contactor window. If the contactor 102 is firmlypressed on the skin, ink may not escape the contactor window. Thecollected ink can be controllable from the contactor window. Thecontactor systems described herein may be used without tattoo frames. Insome embodiments, the contactor system may be pressed against the bodypart by a robotic arm or gantry system, controlled by the application ofan appropriate range of force (0.1-50 N) and/or displacement (0-10 cm)against the skin. Independent of the method of maintaining contact withthe body surface, the contactor system may (i) stabilize skin distance,(ii) prevent runoff liquids, (iii) house an integrated suction system incontact with the skin surface, and/or (iv) house integrated movementdetection sensors for safety. For example, in one embodiment, thecontactor system may be attached to the end of a robotic arm, in orderto maintain a stable skin distance that is fixed relative to thetattooing head, upon approaching and landing on a desired part of thebody.

The controller 108 can be a computing device with one or more displaysfor displaying artwork, tattoo designs, stenciling, tattoo needle paths,tattoo session information (e.g., length of session, costs, color ofinks to be applied, etc.), and/or visualization of artwork to beapplied. In some embodiments, a display 99 can provide visualization ofartwork selected by the client. The client can input locationinformation such that the system virtually applies the tattoo usingaugmented reality or other visualization techniques. A user can specifya location by overlaying an image of the design on an image of theirskin (e.g., via a live feed from their camera, a previously capturedimage, etc.). The system can then use computer vision techniques toidentify position and orientation of the design in relation to, forexample, the body party and/or one or more skin features, such asexisting tattoos, moles, hairs, wrinkles, blemishes, etc. The positionand orientation of the design, in relation to these skin features, canthen be stored (e.g., stored by controllers 108 and/or 109), allowingthe tattooing system 90 to recognize these skin features and apply theselected design with the same position and orientation characteristics.

If a color tattoo will be applied, the tattooing system 90 canautomatically select recommended colors based on the tattoo design, skincharacteristics (e.g., skin color, skin tone, etc.), and/or other tattooparameters. For example, the tattooing system 90 can have apre-determined mapping of skin characteristics to preferred orundesirable tattoo characteristics that it can use to make suggestionswhen a user identified to have such a skin characteristic selects adesign with undesirable tattoo characteristics or without preferredtattoo characteristics. In some implementations, this mapping caninclude corrective measures, such as a change in color or tattooposition when such a suggestion is made. The client and/or operator canselect the size the tattoo, color the tattoo, place in the tattoo,and/or parameters based on the displayed information. The display 99 canbe a touchscreen to enable convenient input. Additional details ofselecting, viewing designs, and input information about tattoos arediscussed in connection with FIG. 9. A stencil can be applied to thecustomer to review the design's positioning on the skin before startingthe tattooing operation. Positioning of the design and/or stencil mayalso be reviewed using augmented reality. In some embodiments, a finaltattoo design can be overlaid on a camera image or live video, based onthe positioning and deformation of the applied stencil on the imagedetected by machine vision.

Referring to FIG. 1B, a container 121 can be fluidically coupled to aneedle structure (e.g., needle structure 140 of FIG. 2). The tattooingapparatus 100 (FIG. 1B) can include a fluidic system having one or morelines (e.g., hoses, multi-lumen conduits, etc.), pumps (e.g.,peristaltic pumps, diaphragm, piston or centrifugal pumps, piezoelectricpumps, etc.), valves, manifolds, filters, sensors (e.g., pressuresensors, flow sensors, etc.), and other fluidic components. Thecontainer 121 can be a bottle, cartridge, or other container suitablefor holding fluid. In some embodiments, the tattooing apparatus 100 isconfigured to hold multiple containers to apply color tattoos, avoidingdowntown for replenishing fluid, or the like. The number of containers,volume of the containers, and configuration of the fluidic system can beselected based on the desired system functionality. The tattooing system90 can include any number of pumping mechanisms, such as peristalticpumps, diaphragm, piston or centrifugal pumping, piezoelectric pumping,capillary effects and so on. The configuration of the pumping mechanismscan be selected to provide the desired volume of ink to the needle tipwhen the tattoo is performed while not overflowing the needle reservoir.

FIG. 2 is a schematic isometric view of an embodiment of the tattooshuttle 104 of the tattooing apparatus 100. The tattoo shuttle 104 caninclude an arm 110, a contactor 120, a target site analyzer or machinevision device 130 (“machine vision device 130”), and a needle structure140. The X gantry 105 can connect the tattoo shuttle 104 to thecantilevered tattoo machine 101. In some embodiments, the tattoo shuttle104 can slide relative to the cantilevered tattoo machine 101 on the Naxis. The sliding relation may be provided by one or more springmechanisms, linear slides, rail systems, or the like. The arm 110 can bepart of the tattoo shuttle 104 and may be connected to the tattooshuttle 104 through the Y gantry 106 and precision X gantry 107. The Ygantry 106 may be a mechanical gantry movable along the Y axis. Theprecision X gantry 107 may be a mechanical gantry movable along the Xaxis. Both the Y gantry 106 and the precision X gantry 107 may connectthe arm 110 to the tattoo shuttle 104 to allow movement of the arm 110along the X and Y axes.

The arm 110 may be further configured to hold actuators, such as, forexample, a needle motor 111, an actuator 112 (e.g., zero stepperactuator), and an arm solenoid actuator 113. In one embodiment, needlemotor 111 may be an electric motor configured to generate the rotationalmovement of a cam 114. In other embodiments, needle motor 111 maycomprise of any other type of motor or method for generating rotationalmovement of cam 114. The needle motor 111 may also be connected to motorgantry 115, which is a structure that holds the needle motor 111 thatcan be lowered or raised by the action of the arm solenoid actuator 113.In one embodiment, the actuator 112 may be a stepper motor connected tothe arm 110 and configured to set a needle maximum extension. In oneembodiment, the arm solenoid actuator 113, may be a solenoid connectedto the arm 110 which controls a position of the motor gantry 115. Thearm 110 may be configured to be movable in the X, Y, and Z axes. In oneembodiment, the arm 110 may also hold the needle structure 140.

In one embodiment, the contactor 120 may be a disposable component incontact with the skin and monobloc with the tattoo shuttle 104. Thecontactor 120 may comprise of a variety of shapes and sizes and may beconfigured to flatten the skin and keep excess ink and other fluid(s)from spreading. For example, in one embodiment the contactor 120 isgenerally rectangular and/or flat with a rectangular window configuredto flatten and expose a portion of the skin to the needle structure 140.In other embodiments, however, the contactor can be shaped and/or sizedin accordance with a contour of the area to be tattooed. In oneembodiment, the contactor 120 is in contact with the skin and can movealong the X axis. While in contact with the skin, the contactor 120 mayapply a nominal force on the skin in the N axis direction, as referencedin FIG. 1B. The amount of force can vary, but in one example, the forcemay be between 1 lb and 10 lb. In some embodiments, the applied forcecan be less than about 1 lb, 2 lb, 3 lb, 4 lb, 5 lb, 6 lb, 7 lb, 8 lb, 9lb, or 10 lb. The contactor 120 may contain a window that exposes aflattened area of skin. The window may comprise of a width and a lengththat varies in direction and magnitude. For example, the window lengthmay be in the Y axis direction while the window width may be between 0.5mm and 5 mm. In another embodiment, the window may span from one end ofthe tattoo frame 103 to the other end of the tattoo frame 103 in the Yaxis direction.

The machine vision device 130 can be part of the shuttle 104 or aseparate component of the apparatus 100. The machine vision device 130can include an imaging device 131 and a lens 132 and be configured toobtain one or more images of a portion of skin. The imaging device 131may be, for example, one or more sensors, cameras, or image captureelements connected to the lens 132. In other embodiments, the imagingdevice 131 may be a plurality of sensors or a digital camera. In oneembodiment, the lens 132 may be a telecentric lens, such as a set ofoptical elements normal to the XY plane and focused on the window ofcontactor 120. The orientation of the telecentric lens 132 and its fieldof vision for the machine vision device 130 may vary, however. Forexample, the field of vision may span the entirety of a tattoo field. Inanother example, the machine vision device 130 may be kept at a fixeddistance from the contactor 120 to keep the skin in the depth of fieldof the telecentric lens 132. Additionally, in another embodiment, thelens 132 may be fixed with respect to the contactor 120 and the tattooshuttle 104. In some embodiments, the machine vision device 130 mayadditionally include an illumination system such as a light source (notshown). The illumination system may be positioned such as to minimizespecular reflection toward the machine vision device 130.

In some embodiments, the needle structure 140 can include a needlecartridge 141, needle 142, needle piston 143, needle spring 144, plunger145, and cam 114. The needle cartridge 141 may be a disposable componentholding a tattoo needle and composed of the needle spring 144, needlepiston 143, and needle 142. The needle cartridge 141 may be connected tothe tattoo shuttle 104, but alternatively, may be configured to beremovably coupled with the tattoo shuttle 104. The needle 142 may be astainless-steel needle composed of a plurality of tapered and sharpenedrods brazed together. The configuration of the needle 142 and cartridge141 can be selected based on the tattoo to be applied, characteristicsof the subject's tissue, or the like.

The needle piston 143 may be a plastic rod holding the tattoo needle142. The needle spring 144 may be a plastic membrane connected to theneedle piston 143. The plunger 145 may be a metal rod joined to the cam114 and the needle piston 143. The cam 114 may be a metal cam with afixed eccentricity transforming, together with the plunger 145, therotational movement of the needle motor 111 to a linear movement of theneedle piston 143, and subsequently the needle 142, along the Z axis.Alternatively, the components of the needle structure 140 may be of anysuitable material aside from those mentioned for the embodimentdescribed. In other embodiments, the components such as the needle motor111 and cam 114 may be replaced by any other method suitable forgenerating movement (e.g., linear movement) of the needle piston 143.

Methods for Applying Tattoos

FIG. 3A is a block diagram of a tattooing process 300 in accordance withan embodiment of the disclosure. The tattooing process 300 can be usedto apply tattoos (e.g., micro tattoos, dotwork, blackwork tattoos,realism tattoos, fine-line tattoos, etc.) and other permanent ortemporary artwork. Although certain features of the method 300 aredescribed with respect to embodiments of FIGS. 1A-2, it will beappreciated that the process 300 can be performed using any of thesystems, devices, and technology discussed with respect to FIGS. 5A-19.

In step 301, a portion of skin that will receive the tattoo can beshaved and cleaned. For example, an operator (e.g., operator 72 of FIG.1A) can manually clean and shave the site. The preparation protocol canbe selected based on the subject's health, characteristics of the targettattoo site (e.g., amount of hair, skin condition, size of target site,appearance of skin, etc.), characteristics of tattoo site (e.g., a flatregion, a curved region, etc.), or the like.

In step 302, a stencil can be applied to the portion of skin. Thestencil may be a set of dots printed on transfer paper that serves as apositional fiduciary to identify the deformation of the skin during thetattooing process. Techniques for identifying deformation of the skinare discussed in connection with FIG. 7. In one embodiment, the stencilmay also contain an outline or contours of the design for the subject topreview the placement of the tattoo. An algorithm may be used togenerate the contour from a design. Generation of contours or outlinesof a tattoo design is discussed in connection with FIG. 19. The stencilmay also comprise of a subset of dots with variable shapes or patternsto encode dot positions. The size, shape, color, and density of the dotsof the stencil may vary. In one embodiment, the dots may correspond todot positions on a dot parameter table. The stenciling can be applied byan operator or the tattoo apparatus. In some embodiments, a stencilingprotocol can be generated based on the tattoo design. The stencilingprotocol can include, without limitation, generating a pattern ofreference features (e.g., temporary dots) generated to assist withmachine vision-based positioning of a tattoo device. For example,spacing between reference features can be reduced to apply microtattoos. In some embodiments, automated and manual procedures are usedto prepare the treatment site. For example, the stencil can be manuallyapplied to the treatment site, and the tattoo apparatus can then apply apattern-encoded set of reference features to the target site. In someembodiments, the stencil may be transferred to the skin by theintermediary of a flat substrate, such as paper, onto which the stencildesign may have been printed using an appropriate dye to temporarilystain the skin. These stencil and reference features can be used todetermine where to puncture the skin to produce the desired tattoo. Themanual and automated steps of a stenciling protocol can be selectedbased on the functionality of the tattooing apparatus. In someembodiments, the natural features of the skin may be used as a referencefiducial instead of a stencil.

After application of the stencil 302, the subject and tattooingapparatus operator may check the stencil 303. The subject may review thestencil application and either approve or disapprove of the designplacement. The operator may review the stencil for quality ofapplication. In some embodiments, for example, a stencil depositionshould be such that a transfer of fiducial marks is adequate to performa machine vision algorithm and the pattern formed by the fiducial marksshould not be substantially deformed when the skin is relaxed. If thestencil appears misplaced or the application is not accepted by theclient, the stencil application step may be repeated until accepted. Inone embodiment, the tattooing apparatus 100 may review the stencil forplacement and/or application, automatically using, for example, one ormore machine learning models trained to review stencil results forquality (e.g., could use past accepted and rejected stencil applicationsas training data).

Following approval of the stencil application, lubricant is applied instep 304. A variety of suitable lubricants with different viscositiesand hydrophobic properties may be suitable for use. For example, alubricant with a viscosity between 10 cps and 500 cps with hydrophobicproperties to increase the contact angle between ink droplets and skinmay be used. The lubricant can be chosen such that the type andviscosity of the lubricant may allow it to protect the epidermis topsurface from being stained by ink and/or increase ease of removal of theink and/or lubricant by suction via suction system (e.g., suctionsystems 150, 450). In some embodiments, the lubricant may be appliedautomatically by the machine 100 when and where it is suitable by anintermediary of a fluidic system.

Tattoo machine preparation 305 can occur after the lubricant is appliedin step 304. Some steps of the machine preparation 305 may be performedbefore the arrival of the client and/or after selection of the tattoo tobe applied. The tattoo machine preparation step 305 may comprise of avariety of activities and may differ between tattooing processes 300.Referring now to FIGS. 1B and 3A, in one embodiment, the operator maybegin by mounting an appropriate tattoo frame 103 (FIG. 1B), whichcorresponds to the area of the body to be tattooed, such that the tattooframe 103 is in contact with the skin to isolate the area where thetattoo is to be applied. In one embodiment, the tattoo frame 103 may beconfigured such that an operator may mount one or more sterileelectrodes 117 (two illustrated in FIG. 1B) to the tattoo frame 103. Theelectrodes 117 may comprise of the same or different electrodes capableof making at least one type of measurement. For example, the one or moreelectrodes 117 may be a plurality of electrodes for measuring galvanicresponse. In one particular embodiment, the one or more electrodes 117may already have been mounted on the tattoo frame 103 prior topreparation. The number, type, and positioning of electrodes 117 mayvary depending on the desired measurements and configuration. In oneembodiment, for example, one electrode 117 may be mounted at a locationmore proximate to the contactor 120 than another electrode 117. In otherembodiments, electrodes 117 may be placed or located on other parts ofthe tattooing apparatus 100 and/or on the subject's body. Followingmounting the tattoo frame 103, the operator may then proceed to isolatethe tattooing area from the rest of the body and tattooing apparatus 100by mounting, for example, a sterile membrane and/or bag system.

Referring again to FIG. 2, the operator may then mount a sterilizedneedle cartridge 141 and an ink cartridge (not shown) to the tattooingapparatus 100. In one particular embodiment, the needle cartridge and/orink cartridge can be coupled to the needle structure 140. Example needlecartridges are discussed in connection with FIGS. 10A and 10B.

With continued reference to FIGS. 1A-3A, after mounting the needlecartridge 141 (FIG. 2) and ink cartridge, the operator may then mountadditional components of the machine vision device 130 (FIG. 2) to themachine. In one embodiment, the additional components of the machinevision device 130 may be a sterile imaging device 131 and visionviewport. The vision viewport may be a sterilizable viewport suitablefor camera vision. Other embodiments may comprise fewer or morecomponents. In one particular embodiment, the machine vision device 130may already contain all necessary parts and preparation may compriseonly of sterilizing the parts.

Additionally, the operator may mount a suction system 150 (FIG. 2) tothe tattooing apparatus 100. The suction system 150 may comprise of oneor more conduits, lines, pumps, valves, nozzles, containers, filters,and/or other components. In one embodiment, the suction system 150 maybe, for example, a sterile, single usage suction line comprising of anozzle, tubing, a microperforated air filter, and a liquid trap. Inother embodiments, the suction system 150 may be reusable and/orcomprise more or fewer parts. In one particular embodiment, the suctionsystem 150 may already be mounted to the machine and preparation maycomprise only of sterilizing the suction system 150.

Following preparation 305 of FIG. 3A, the tattooing apparatus 100 maycalibrate itself, or may be calibrated by the operator, in tattoomachine calibration step 306. Tattoo machine calibration 306 maycomprise of running all or some of the actuation systems and sensors toevaluate nominal functioning. Referring now to FIGS. 2 and 3A, thetattooing apparatus 100 may perform zero reference calibration bymeasuring its range. In one embodiment, range can be measured throughusing a plurality of end range sensors, encoder sensors, etc. tocalibrate the position of the needle 142 in space. Running all sensorsmay also comprise of running components of the machine vision device 130such as the imaging device 131. In one embodiment, calibration maycomprise of using an algorithm to run a diagnostic of the sensors andimaging device 131. Additionally, a conductivity test may be performedto confirm connection of one or more electrodes to the machine and/or tothe skin. The calibration step 306 may also comprise of using areference pattern in the tattoo frame 103 to confirm camera vision andillumination. In various embodiments, the calibration step 306 maycomprise more or fewer than the preceding steps as well as other stepssuitable for appropriate calibration of the machine for the tattooingprocess. In some embodiments, the calibration step 306 can be omitted.In some implementations, parts of the calibration process can beperformed at other times, such as when the tattooing apparatus 100 ispowered on, prior to the stencil application, prior to the tattoomachine preparation step 305, or other suitable time.

Following calibration step 306 of FIG. 3A, the operator may then proceedto positioning step 307, which may comprise of positioning the tattooingapparatus 100 appropriately relative to the area to be tattooed andconfirming a proper positioning. Referring now to FIGS. 1A-3A,positioning the tattooing apparatus may comprise of selecting theappropriate size of tattoo frame 103, orienting the subject and area tobe tattooed relative to the rest surface 102 and tattoo frame 103, andaligning a machine tattoo zone with the area to be tattooed by rotatingand/or repositioning the tattooing apparatus 100 and/or subject to matcha location of the tattoo. An operator, or the machine itself, may thenconfirm appropriate positioning by using the machine vision device 130,e.g., to determine that identified skin features and/or stenciling arepositioned to allow the tattooing apparatus 100 to apply the selecteddesign with the previously determined position and orientationcharacteristics. In one embodiment, the machine vision device 130 mayuse a machine vision algorithm to identify each dot position in astencil and compare it to a vector-based stencil design. The position ofthe stencil with respect to the tattoo frame 103 may be validated aswell as whether there is a correct transfer of dots for calculating skinstretch and/or deformation. If the stencil deposition is such that thetransfer of fiducial marks is inadequate to perform the machine visionalgorithm and/or if the tattoo frame 103 is not properly positioned, anoperator error is generated suggesting either reapplication of thestencil or realignment of the tattoo frame 103. In one embodiment, forexample, an error may be generated if a sufficient number of dotscorresponding to an aspect of the design (e.g., design contour) do notmatch. The machine vision device 130 may include one or more controllersstoring the machine vision algorithms. Alternatively, separatecontrollers (e.g., controllers 108, 109) can perform the machine visionalgorithms.

After confirming that the machine is properly positioned, skin punctureproperty acquisition step 310 may then be performed by the machineitself, or the operator. FIG. 3B illustrates an expanded block diagramof an embodiment of the skin puncture property acquisition step 310. Forexample, the skin puncture property acquisition step 310 may comprise ofperforming a series of punctures distributed around the area to betattooed with no ink. In some embodiments, multiple punctures may beperformed for each position. The machine may be set to a safe depthsetting such that the epidermis and dermis are punctured in the process.For each of the positions, information about the puncture operation isrecorded with one or more sensors, which may be redundant forreliability. In one embodiment, the one or more sensors may include, butare not limited to, a load cell sensor, an accelerometer, an encoder, aset of galvanic electrodes, and a vibration sensor. The load cell sensormay be located on the needle plunger 145 of FIG. 2 and may be configuredto measure a load applied to the plunger 145 during tattooing. Theaccelerometer may be on the plunger 145 and may be configured to measurean acceleration of the needle 142. The encoder may be located on theneedle motor 111 and may be configured to measure an angular position ofthe needle structure 140. The set of galvanic electrodes may beconfigured to measure an impedance of the needle 142 to skin contactrelative to an impedance of the skin alone. The vibration sensor may beconfigured to measure a vibration of the contactor 120. The locations,quantities, and types of sensors may vary depending on the particularembodiment and process.

Referring now to FIGS. 2 and 3A, the sensors can measure properties ofthe needle/skin system and precisely identify when skin puncture eventsoccur during one rotation of the needle motor 111 (FIG. 2). Certainfactors related to the needle/skin system may influence the complexbehavior of the skin in terms of deformation and failure. For example,needle sharpness, number of needles, needle gauge, and speed may all beinfluencing factors. Additionally, certain skin characteristics such aselasticity, impedance, thickness, etc. may also be influencing factors.By measuring when the different layers of the skin fail (colloquially,when puncture occurs), the configuration of the machine can be varied byvarying needle 142 (FIG. 2) extension out of the cartridge 141 andneedle motor 111 speed such that a force applied to the skin isminimized and a maximum extension of the needle 142 can be insured. Onerational is that an ultimate position of a tip of the needle 142 shouldbe in the papillary zone of the dermis for optimal tattooing (e.g., toprevent or limit scattering, subdermal diffusion, etc.).

Referring again to FIG. 2, one or more of the exact elastic property ofthe skin, thickness of the skin, sharpness of the needle 142, and/orexact force transfer from the needle 142 to the skin may be unknown.Puncture tests can be performed on the skin to determine one or moremachine settings. The puncture events can integrate some or all theparameters needed to pilot the needle 142 without having to measure skincharacteristics, needle characteristics, and/or machine characteristics.In particular, the skin may be susceptible to deflection andcompression, which are the primary determinates of the layer of tissuein which the needle reached. The deflection of the skin can beproportional to the distance traveled by the needle 142 between firstskin contact and epidermis puncture. The measurement of skin contact,initial puncture, needle displacement and/or needle extension (e.g.,maximum needle extension) can be used to pilot the height of the motorgantry 115 compared to the needle cartridge 141, which in turn can setthe ultimate extension of the needle 142 and control the depth ofpiercing of the tissue (i.e., the puncture depth). The puncture depthcan be the depth of a single needle or an average depth of a pluralityof pins (e.g., a bundle of pins) of a tattoo needle. In someembodiments, the ultimate extension can be the maximum depth of a bundleof pins of a tattoo needle. For reliability, each position may besubjected to a train of punctures ranging from 1 to 50 punctures and themeasured metrics may be averaged over multiple punctures. In someembodiments, the sensing strategy may be configured to detect a depth ofthe needle during a puncture event based at least in part on arelationship between a force applied to the skin and the depth of thetissue layer interface. Another embodiment of the current sensingstrategy may be configured to detect the depth of the needle during apuncture event based at least in part on the contact conductivity(including changes in contact conductivity) of the needle against theskin, relative to the conductivity of the skin alone. The detectedpuncture event may be, for example, the failure initiation of theepidermis during the needle transition from surface of the epidermis tothe epidermis to dermis layer interface. In some embodiments,determination of a proper height setting, needle extension, punctureevent depth, and/or predicted depth of ink deposition may be based atleast in part on the force applied to the skin. In other embodiments,determination of a proper height setting, needle extension, punctureevent depth, and/or predicted depth of ink deposition may be based atleast in part on the contact conductivity, such as the variation of thecontact conductivity of the needle against the skin, relative to theconductivity of the skin alone as exemplified and discussed inconnection with FIGS. 12A and 12B. The puncture events can be identifiedby analyzing the conductivity data. For example, puncture events can bedetermined based on, for example, extrema of the conductivity curveobtained by the galvanic sensor, as well as the extrema of its firstderivative with respect to time, as exemplified and discussed inconnection with FIG. 12B.

For each location on the skin tested, the target needle 142 extensionparameter (e.g., maximum displacement) for the execution of a dot ofacceptable quality may be calculated using one or more algorithms basedon collected data associated with initial or first contact, skinpuncture, and maximum depth as exemplified and discussed in connectionwith FIGS. 12A and 12B. The data for each of the test positions can bestored. In one embodiment, for example, calculating the correct ultimateneedle extension (e.g., maximum displacement of the needle or puncturedepth) may be based at least in part on: a relative impedance of theskin/needle contact compared to the skin impedance as measured by thegalvanic electrodes, a force magnitude, needle speed, needleacceleration, an angular position correlating to a needle 142 positionat a certain time, or an algorithm that uses measured puncture eventssuch as contact with skin, initial epidermis failure, additional layerfailure, or ultimate needle 142 position to evaluate the maximum needle142 position that results in a correctly applied tattoo dot. Thecollected data and metrics collected for each position tested can bestored, aggregated, analyzed, etc.

In one embodiment, an algorithm for predicting the ultimate needle depthin relaxed skin can be based on the sum of a weighted polynomial or anyother set of relevant basis functions of the sensor measurements.Experimental calibration can be used to obtain the coefficientsassociated with each polynomial term. The depth prediction is thencompared to the desired depth on relaxed skin to issue a change ofheight of the needle 142. An algorithm can be used to predict the depthof ink deposition based on the needle position at contact, the needleposition at initial puncture, the needle position at max extension, theneedle position when exiting the skin, and/or the angle of attack of theneedle. Illustrative diagrams of example puncture events, for referenceare discussed in connection with FIG. 12A. The extrema of the output ofthe galvanic sensor can be correlated to each example of the punctureevents as discussed in connection with FIG. 12B.

Puncture events can be used to calibrate for variation of needle lengthsdue to, for example, variation in needle manufacturing. During oneoscillation, the needle starts from its uppermost position. Then theneedle contacts the skin. Following this, the needle punctures variousskin layers until the needle reaches its lowermost position, the maximumextension. Longer needles will touch the skin earlier while shorterneedles will touch the skin later in the cycle. Similarly, the skinheight may vary within the contactor window by forming a bulge. Byvarying the needle extension, the tattooing system can compensate forvarious needle lengths and skin positions/heights by, for example,raising the needle (e.g., raising with respect to skin surface) tocompensate for longer needles and/or higher skin height or lowering theneedle to compensate for shorter needles and/or lower skin height. Insome procedures, needle extension may be varied to maintain a consistentdistance measured between skin contact and needle at maximum extension,and thus maintain a consistent depth of ink deposition. Measuring thedistance between the tattooing head and the skin surface, for exampleusing distance sensors (e.g. based on time of flight, light projection,haptic sensors, etc.) does not account for variations in needle length,or other geometric variations of needle cartridges.

Referring again to FIG. 3A, the skin puncture property acquisition 310can include determining an initial deflection to puncture, which is thedistance traveled by the needle from the time of first skin contact tothe time of initial skin puncture. A distance traveled to puncturecorresponding to the difference between a position of initial skinpuncture and a position at skin contact can be derived. This distance isthe initial deflection. Similarly, a distance traveled in the skin tomax extension, which is the max extension, is the difference between aposition at a predetermined or maximum extension and the position atcontact. Because the skin deforms during puncture, the predetermined ormaximum extension is not directly linked to the depth of ink deposition.Similarly, the initial deflection can relate only to the deflectionuntil puncture, not the deflection at maximum depth, which is the totaldeflection. The total deflection may be assumed in the algorithm to havean affine relationship with the initial deflection. A more complexrelationship between total and initial deflection may be devised basedon weighted polynomials or any set of relevant basis functions. Aprediction of the depth of ink may be obtained by deriving an affinerelationship (or another appropriate mathematical fit) between the depthof ink and the difference of the max extension and the total deflection,accounting for the necessary trigonometric relations due to the needleangle with respect to an axis normal to the skin's surface. In someembodiments, the desired ink location for human skin may lay between 0.5mm and 1.2 mm deep on relaxed skin. In some embodiments, the change inneedle 142 height setting can be calculated to be the difference betweenthe predicted depth and the desired ink location. From the calculationsand measurements, a corresponding map of the tested positions on theskin containing the measured and/or calculated metrics such as, forexample, a needle height, a needle extension, and/or a predicted depthof ink deposition may be generated and stored. The depth of the ink maybe used as a non-visual metric to evaluate dot quality. A method for theanalysis of puncture parameters and/or determination of puncturesettings based on training data explicitly introducing dot aestheticquality is discussed in connection with FIG. 13.

In one embodiment, each position on the skin may be identified by themachine vision device 130 (FIG. 2) to confirm an absolute skin location.In normal operation of the machine, the skin is stretched resulting in aplanar elastic deformation. This in turn means that the position of themachine needle 142 does not necessarily match with the position on theskin due to skin stretch. The machine vision device 130 may be used toacquire at least one image of the skin and a digital image correlationor other image analysis method may be used to calculate a skindisplacement field. An embodiment of this method is presented inconnection to FIG. 7. This displacement field may be used to calculatethe true location of the needle 142 on the skin such that the set ofsensor data used for defining a proper needle extension for each skinlocation is in a relaxed skin frame of reference. An algorithm thatcalculates skin deformation may be used to identify the position of theneedle 142 from processing of at least one acquired image of the skin.Additionally, information about skin puncture in a skin positional frameof reference may be recorded in a database. The database may also, forexample, record the prescribed needle extension at the sensed punctureposition in an undeformed positional frame of reference of the skin.

Once all the prescribed positions designated for testing of the skinhave been punctured, the punctures analyzed, and the prescribed needleextension for these positions saved, an interpolation algorithm may beused to interpolate the needle extension for all the dot locations thatare part of the tattoo design. This allows evaluation of the properheight setting of the machine. The interpolation algorithm may be, forexample, an algorithm that interpolates the prescribed needle extensionfrom the test puncture points to the dot positions corresponding topositions not tested. The interpolation can include determining a depthplane for all punctures that is a fit to the saved positions and appliesa smoothing function between them.

FIG. 3B illustrates an embodiment of the skin puncture propertyacquisition suitable for the step 310 that is shown in FIG. 3A. The skinpuncture property acquisition step 310 can begin, in step 311, atposition i. A puncture train is then run at position i, which mayconsist of at least one puncture 312. Data measured from the puncturesby at least one sensor may then be analyzed for each position i 313.Certain puncture metrics may be calculated and a machine setting forposition i may be determined based on the measured data andappended/recorded for position i as in step 314. For example, themachine setting may be based at least in part on any of the calculationsas described. In one embodiment, the machine setting may be at least aproper height setting, needle extension, and may include parametersbased upon puncture events, tissue layer interface depth, or predicteddepth of ink deposition. Steps 311-314 may be repeated for all testingpositions 1 to n according to test position table 315. The appended andrecorded machine settings may then be used to update dot parameters of adot parameter table 316, which may be representative of the tattoodesign. The updated dot parameter table 316 may comprise of one or moredot positions, wherein at least some of the dot positions may correspondto the tested positions of test position table 315. In one embodiment,there may be more dot positions than tested positions and machinesettings may be interpolated for the dot positions that do not have acorresponding test position 317. For example, machine settings can beinterpolated for dot positions that do not correspond to testedpositions of the test positions table 315 based on the appended andrecorded machine settings for tested positions. In another embodiment,there may be the same number of dot positions as tested positions, andthe tested positions may correspond to all dot positions in the updateddot parameter table 316.

Referring now to FIGS. 3A and 3C, the tattoo application process step320 follows completion of skin puncture property acquisition step 310.FIG. 3C shows an expanded block diagram of an embodiment of the tattooapplication process step 320. In one embodiment, tattoo applicationprocess step 320 may be based at least in part on the updated dotparameter table 316 (FIG. 3B) with corresponding appended and recordedmachine settings. In one example, a map of prescribed needle depths maybe defined based at least in part on the updated dot parameter table316. Once the map of prescribed depths is defined, the tattoo machinerepositions itself at the beginning of the tattoo field. Referring nowto FIG. 2, the imaging device 131 (FIG. 2) may be used to scan a tattoowindow i. In some embodiments, tattoo window i may correspond to awindow of the contactor 120. The suction system 150 may be turned on. Amachine vision deformation algorithm and a position algorithm may beused to evaluate the deformation of the skin as well as the position ofthe contactor 120, respectively. Algorithms for detecting the skindeformation and contactor position are discussed in connection withFIGS. 6A, 6B, and 7.

The deformation of the skin may be used to identify the positions of thetattoo dot prescribed in the reference undeformed vector-based graphics(one or more vector graphics) in the frame of reference of the contactor120. The gantry movement of the tattoo shuttle 104 (FIG. 1B) isprescribed for realizing the tattooing of all dots that are within acentral subset of the tattoo window i from one side to the opposite sidein the Y direction. Ink may be added to the needle cartridge 141 and/orink cartridge by an ink recharging process. This ink recharging processmay be repeated mid tattooing of the window i if the level of ink in theneedle cartridge 141 and/or ink cartridge is insufficient to tattoo allthe dots present in the tattoo window i.

With reference to FIGS. 1B, 2, and 3B, the needle motor 111 is startedto reach expected tattoo speed and to wet the needle 142 tip with ink.The zero-stepper actuator 112 is set for the needle extension of a firsttattoo dot in the window i. The X gantry 105 and Y gantry 106 areactuated to position the needle 142 tip over the first dot location. Thearm solenoid actuator 113 is energized to drop the motor gantry 115 downsuch that the needle piston 143 is compressed by the needle plunger 145and that needle oscillation occurs at the Z reference set by thezero-stepper actuator 112. A reason for the arm solenoid actuator 113 isto rapidly engage the needle 142 with the skin without the needle motor111 starting inertia. The needle 142 oscillates between a high positionand a low position defined by the maximum needle extension controlled bythe zero-stepper actuator 112. A number of punctures for a single dotmay be set by a design file and may vary the dot size and color density.For example, the number of punctures for a single dot may be between 1puncture and 50 punctures. An encoder sensor system and a galvanicsensor system may count the number of punctures and trigger the armsolenoid actuator 113 to raise the arm 110 when the prescribed number ofpunctures for that dot is reached (or calculated to be reached by takinginto account transient effects related to actuation time).

The machine vision device 130 (FIG. 2) can be used to confirm correctdot application at that time. Once the arm 110 is confirmed to be raisedby the sensor and algorithm, the needle 142 is not in contact with theskin and the gantry actuation is engaged to position the machine to theappropriate X and Y location and the calculated Z reference of the nextdot to be tattooed. This tattoo process is repeated for all positions tobe tattooed identified by the machine vision algorithm within the tattoowindow i. The suction system 150 may be on throughout the process andmay collect superficial drops of ink to improve the visibility for themachine vision device 130, or may be turned on when suction is necessaryto remove fluids . . . .

Once a selected number (e.g., all the dots in the tattoo window i) aretattooed, a drainage system, which may comprise part of the suctionsystem 150, may be triggered to remove the chance of ink dropping fromthe cartridge because such ink drops compromise the imaging of the skinby machine vision device 130. The needle motor 111 may then be turnedoff. The contactor 120 may be actuated forward in the X direction by afraction of the window width, such that each new window of tattooing i+1may overlap with at least a portion of a preceding window i. The forwarddirection is decided based on the natural growth direction of thetissue, which is generally from a base to extremities of the limbs, orfor trunk tattoos, in the direction of gravity. The tattooing of thecontactor 120 window in the new position i+1 repeats the same steps fromthe preceding tattoo window i and is reiterated until the end of thetattoo field is reached m. Once this is the case, the tattooingapparatus 100 may be put in a safe position and all actuators may beturned off.

FIG. 3C illustrates an embodiment of the tattoo application processsuitable for step 320 shown in FIG. 3A. Referring now to FIG. 3C, thetattoo application process step 320 may begin at window position i as instep 321. The machine vision device 130 may then acquire at least oneimage of the stencil and store it in a raw image database as in step322. A position algorithm may be used to calculate a position of thecontactor 120 (see FIG. 2) and window i. Similarly, a deformationalgorithm may be used to calculate a deformation of the skin at window ito identify the positions of the tattoo dot prescribed in the referenceundeformed vectoral graphics in the frame of reference of the contactor120. Both the position algorithm and the deformation algorithm may takeinto account at least a stencil reference image 324 when calculating theposition and skin deformation in step 323. Algorithms for calculatingthe position and skin deformation are discussed in connection with FIGS.6A, 6B, and 7. Based at least in part on a tattoo design and thecalculated position and skin deformation of step 323, the tattooapplication process step 320 may continue by evaluating one or morepositions k where tattoo dots are to be placed 325. The tattooingapparatus 100 may then tattoo all one or more positions k in the windowi as in step 326 based at least in part on the updated dot parametertable 316. At step 327, if km, the process returns to step 312. Steps321-326 may be repeated for all window positions 1 to m within windowposition table 327.

Referring again to FIG. 3A, after tattoo application process step 320 iscompleted, the tattooing apparatus may be removed as in step 308. Theoperator may remove the tattoo frame (e.g., frame 103 of FIG. 2) fromthe tattoo zone such that the subject may be freed from the tattooingapparatus 100 (FIG. 1). Any disposable components may then be removedand disposed of. As used herein, the term “disposable” when applied to asystem or component (or combination of components), such as a needle, atool, or stencil, is a broad term and generally means, withoutlimitation, that the system or component in question is used a finitenumber of times and is then discarded. Some disposable components areused only once and are then discarded. In other embodiments, thecomponents and instruments are non-disposable and can be used any numberof times. In some kits, all of the components can be disposable toprevent cross-contamination. In some other kits, components (e.g., allor some of the components) can be reusable. Following machine removalstep 308 of FIG. 3A, the tattoo area may be cleaned and dressed with aprotection layer in dressing tattoo step 309. Additionally, a variety ofaftercare treatments may be provided. Finally, the surface of thetattooing apparatus 100 of FIGS. 1A and 1B may be cleaned at step 290with a cleaning solution so as to be suitable for a next use. At step291, the process 300 is completed.

FIG. 4 is a block diagram of a tattooing process 330 in accordance withan embodiment of the disclosure. In step 332, a tattoo system can obtaininformation from the target site analyzer, such as a machine visiondevice, sensor, or the like. The information can include, withoutlimitation, one or more images of a portion of the skin, and/or at leastone characteristic of the portion of a subject's skin, skin punctureproperties (step 310 of FIG. 3A), or the like. In step 333, a protocolfor applying a tattoo can be generated based on the received informationand a tattoo design. The protocol can include identifying changesassociated with the skin capable of impacting a visual appearance of thetattoo to be applied. The protocol can include a plan or map of dots orindividual punctures for applying the tattoo, together with applicationparameters for each. The tattoo system can compensate for the one ormore changes (e.g., skin stretch, skin displacement, skin layer(s)thickness changes, etc.) associated with the skin to robotically applyat least a portion of the tattoo. For example, the spacing of the dotscan be adjusted to match skin stretch at corresponding regions. If thesystem detects skin deflection increase, the system can increase the(e.g., puncture depth, maximum needle depth, depth ink is applied,etc.), needle displacement, or other applicator parameters for thatregion. A transformation can be applied to compensate for skin stretch,skin displacement, skin layers thickness changes, etc. For example, thespacing, pattern, and locations of puncture sites can be adjusted basedon changes of the stenciling. A machine learning model can be trained toidentify transformations to produce a tattoo on post-application thatmatches (e.g., geometrically congruent, visually identical to the nakedeye, etc.) the tattoo design. For example, the applied tattoo can beplaced with a deviation from the original design (e.g., averagethreshold deviation between dots and target dot of design less than 50μm, 75 μm, or 100 μm) which may not affect the visual outcome of thetattoo. In some embodiments, a virtual tattoo design can be generated tomatch the variation of measurable characteristics of the skin. Forexample, if the skin is stretched, a stretched virtual tattoo design,stretched stencil, etc. can be configured to compensate for thestretching of the skin. New puncture points and tattooing parameters canbe generated based on the virtual tattoo design. During the tattoosession, numerous virtual tattoo designs can be generated to determinehow to apply a tattoo on skin such that, when the skin is in a naturalstate, the applied tattoo will match the original tattoo design. One ormore machine learning model can be trained to generate virtual tattoodesigns and/or stencils. For instance, a transformation of the tattoocharacteristics such as dot parameter, dot placement, dot density andsuch can be generated such that it would compensate for certain measuredcharacteristics of the skin, such as skin color, variation incoloration, features such as moles and birthmarks and tissue thickness,bone backing and so on. These skin characteristics can be measured bymachine vision and other sensors deployed by the automated tattoomachine. A machine learning strategy can be obtained by first generatinga validated training set. One method may include a professional tattooartist evaluating the type of transformation he/she would perform toobtain a more congruent tattoo for a certain area of the skin. Anothermethod includes applying random transformations to the tattoo and usehumans to evaluate the esthetic quality of virtually projected tattoo onthe body area using augmented reality. Another method is to use agenetical algorithm or other optimized search to generate guesses fortattoo transformation, for evaluation for aesthetic quality by humans onvirtually projected tattoos. A scoring method may be used toautomatically evaluate congruence of the tattoo design aftertransformation. This training set may then be used by a machine learningalgorithm to determine an optimal tattoo design transformation and for alarge set of conditions where some skin characteristics are measurable.Another characteristic of the skin is the shape of the body area, whichis curved and may need a special projection of the flat design (forexample using area-preserving mapping, distance-preserving mapping,conformal mapping, or other projection methods), such that the resultlooks harmonious on curved body surfaces, such as on shoulders, elbows,wrist, etc.

In step 334, the tattoo system can robotically apply at least theportion of the tattoo according to the protocol. The protocol can beused to reduce one or more differences between a selected tattoo designand the tattoo applied to the skin. The stenciling and techniquesdiscussed in connection with FIGS. 5A-6B and 11A and 11D may be used todetermine dot puncture locations. In some computer-implemented methods,the system can apply one or more reference features to the skin andanalyze at least one of the reference features captured in one or moreof the images to evaluate one or more characteristics of the skin todetermine one or more changes in the skin. The tattoo system cancompensate for the one or more changes in the skin to determine puncturesites for applying the pigment.

Methods for Applying and Using Stencils, Image Analyses, and MachineVision Technology

FIG. 5A illustrates constituents of a stencil 338 a and how it may beutilized by a machine vision algorithm, such as machine vision algorithmdiscussed in connection with step 323 of FIG. 3C. FIG. 5A shows oneembodiment of a digital stencil reference that can be stored in memory(e.g., computing unit memory). The stencil reference can include atleast (i) a tattoo design 340 a, (ii) a reference stencil with fiducialmarkers 341 a, and (iii) a pattern 342 a encoded by fiducial markerswhich may be used to guide the machine vision process. The pattern 342 amay be encoded by spatial variations of (a) size, (b) shape, (c) colorand/or (d) presence/absence of fiducial markers. In one embodiment, thepattern 342 a is encoded by (d) presence/absence of fiducial markers.

FIG. 5B depicts an embodiment of the stencil 338 b transferred andviewed on the skin 346. In some procedures, the stencils 338 a, 338 bcan be substantially identical in sizes and/or shapes. The tattoo design340 b on the applied stencil 338 b may be utilized for previewing thetattoo placement, and may be (i) a complete representation of the tattoodesign, (ii) a reduced form of the tattoo design, such as an outline orcontours representing the tattoo design, or (iii) may not be placed onthe applied stencil, at all. The transferred stencil 338 b may stretchand rotate with the skin 346. During tattooing operation, a contactor(e.g., contactor 120 of FIGS. 1B and 2) can move across the skin as instep 321 of FIG. 3C. One possible position of an interrogation orcontactor window 344 b (e.g., detector portion of the interrogation orcontactor window) is shown in FIG. 5B. The contactor window 344 b can bemoved to different locations to analyze and apply dots at differentlocations along the target site.

FIG. 6A shows one embodiment of an image captured by the machine visiondevice through the contactor window 344 b of FIG. 5B. FIG. 6Billustrates features of FIG. 6A detectable using the machine visionalgorithm. Referring now to FIG. 6A, the contactor window 344 b exposesa portion of the skin 346, the applied stencil 338 b, including thefiducial markers 341 b and the encoded pattern 342 b and the tattoodesign on the applied stencil 340 b. The viewable area 345 b may alsocontain other features, such as, previously applied tattooed ink,residual ink, blood, moles, hair, hair roots, skin creases, light glare,etc. These are not shown in the figure for clarity, but may be addressedby the machine vision algorithm, as discussed in connection with FIG. 7.

Referring now to FIG. 6B, the detectable features can include, withoutlimitation, fiducial markers 341 b, the spatial variation of markerswhich constitute a pattern 342 b, deformation field of the skin 347, andother features of the stencil 338 b. The design and use of the stencil(338 a in FIG. 5A, and 338 b in FIGS. 5B and 6A) enables the machinevision algorithm (FIG. 7) to identify the position of the contactorwindow 344 b on the skin 346, map the coordinates of the tattoo dots ofthe design 340 a to their corresponding coordinates on the skin 346where ink shall be deposited, evaluate the skin changes (e.g.,stretching during the tattooing process), and analyze the target tattoosite. A machine vision device (e.g., machine vision device 130 of FIG.2, machine vision device 1600 of FIG. 21, etc.) can utilize a machinevision algorithm to compare the detected reference features and/ortattoo dot locations with a vectoral stencil drawing. In otherembodiments the machine vision algorithm may be performed by acontroller such as a controller (e.g., controller 109 of FIG. 1B orcontroller 1400 of FIG. 20).

FIG. 7 is a flow chart of a method 357 for calculating position and/ordeformation field in accordance with an embodiment of the disclosure.For example, the method 357 can be used in step 323 of FIG. 3C andtechniques discussed in connection with FIGS. 4-6B. In step 358, thesystem can obtain raw images from an image capture device, machinevision device, or other device configured to capture images. Theresolution of the images can be selected based on the resolution of thestencil or other criteria.

In step 361, for a given position of the contactor, images acquired bythe image capture device or machine vision device may be processed(e.g., combined, stitched together, etc.) to form an image of a portionor the entire area exposed through the contactor window (e.g., window344 b of FIG. 6A).

In step 362, the image may be preprocessed to remove, for example, lightglare, reflection, shadows, and/or any variation of illumination on theskin. In one embodiment this step may be performed by colornormalization.

In step 363, the preprocessed image may be analyzed to identify thecolors of (i) the skin and (ii) the stencil fiducial marks as theyappear on the skin. In one embodiment, color identification in step 363may be achieved by principal component analysis (PCA) in the color spaceof the preprocessed image. In step 364, any features that are notrelevant to the deformation and position detection steps (for example,previously applied tattoo ink, residual ink on the skin, blood, moles,hair, hair roots) may be identified on the image and masked to improvethe accuracy of these algorithms downstream (steps 365-367). In oneembodiment, the identification may be based on comparing the color ofthe image pixels to the colors of the mentioned features. In anotherembodiment, these features may be identified using an AI-based featuredetection algorithm.

In step 365, stencil fiducial markers are detected 341 b (FIG. 6B) andtheir size, shape and/or location on the image are determined. Thecontrast and accuracy of this detection may be improved based on, forexample, the colors of (i) the skin and (ii) the stencil fiducial marksas they appear on the skin as identified earlier in step 363. In oneembodiment, the location of the fiducial markers may be identified byfiltering the image with a convolution matrix (kernel), followed byfinding the peaks in the filtered image. The located fiducial markers onthe image may then be further analyzed to identify their size and shape,for example, using other kernels to measure feature size and shape, orby performing pixel-level procedural analysis. In another embodiment anAI-based feature detection method may be used to identify and locate thefiducial markers on the image, based at least on their color, contrast,and/or shape. For example, a convolutional neural network (CNN) may beused for AI-based identification. The training dataset of the CNN mayconsist of a large number of synthetically-generated images, eachshowing a patch of skin with an applied stencil, as would be viewed bythe machine vision device, in various states of deformation, lightillumination, and skin conditions (color, creases, folds, presence ofmoles, hair roots etc.), wherein location, size and shape of thefiducial markers on the image are readily known by the image-generatingalgorithm. The CNN may then be trained by comparing its output to theknown location, size and/or shape of the fiducial markers. In eithermethod, confidence indices may be calculated for each marker detected onthe image, to quantify the confidence in the detected location, size andshape.

In step 366, a deformation algorithm identifies the deformation of theskin 347 (FIG. 6B) within the contactor window 344 b (FIG. 6B) bycomparing the fiducial markers 341 b detected on the deformed image(FIG. 6B) to the fiducial markers 341 a on a reference (undeformed)stencil image 338 a (FIG. 5A). In one embodiment, the deformation of theskin may be represented by a vector field (or by multiple vector fieldsfor different regions) parameterized by a certain number ofcoefficients. An alignment score can be calculated for a givendeformation field produced by a given set of coefficients, which may bebased on the numbers of successfully-aligned and failed-to-align markersbetween the reference image (FIG. 5A) and deformed images (FIGS. 5B-6B),and/or the overall degree of overlap of markers between the referenceand the deformed image. In this embodiment, the algorithm can identifythe correct deformation field by finding the coefficients which producethe highest alignment score. The values of the deformation fieldcoefficients may be restricted to a physically reasonable range tofacilitate the search. If the lattice consists of an ordered arrangement(grid) of fiducial markers, the search may be informed by a periodicityanalysis of the deformed image, for example, using Fourier analysis. Inanother embodiment of the deformation algorithm, a deformation field maybe sequentially or locally constructed based on finding pairs ofneighboring markers on the deformed image and analyzing their relativepositioning with respect to each other, to quantify the localdeformation and rotation of their neighborhood. This sequential analysismay begin from the fiducial marker with the highest confidence index.Different regions of the image may be analyzed separately and stitchedor bridged together by making use of parameterized deformation vectorfields (as in the previous method) to represent the deformation of eachregion. The result may further be processed by using a minimizationmethod fitted to a physically acceptable deformation/displacement field.As a result, the output of step 366 is a deformation field, whichdescribes the displacement of each marker on the deformed image withrespect to the reference image (e.g., image in window 344 a of FIG. 5A),and may be used, along with the detected contactor position (step 367),to map the coordinates of tattoo dots on the reference image to theirtarget coordinates on the skin measured with respect to the contactorwindow.

In step 367, the collection of identified fiducial marks areindividually analyzed to detect the portion of the encoded patternexposed through the window 344 b (FIGS. 6A and 6B). The detected pattern342 b (FIG. 6B) is compared and matched to a similar pattern in thecomplete reference stencil 338 a (FIG. 5A), wherein the best match mayindicate the most likely position 344 a (FIG. 5A) of the contactorwindow with respect to the tattoo design 340 a (FIG. 5A). A confidenceindex of the identified position may be determined, in one embodiment,by comparing the quality of the best match to the second-best match onthe reference stencil or by other methods related to uncertaintyquantification. If the confidence index is found under a criticalthreshold, the machine may attempt to first improve image acquisition byrepeating the cleaning operation of the tattoo process. If theconfidence index is still found lacking, the machine may be stopped bygenerating an operator error. The stopping procedure is detailed below.The above features and techniques can be incorporated or be used withfeatures and techniques discussed in connection with FIGS. 11A-11D.

The machine vision technology discussed herein and in connection withFIGS. 6 and 7 may be used to achieve high-precision tattooing using, forexample, robotic tattoo arms. Machine vision systems may be used todetecting surface topology of the body part where the tattoo is to beperformed, in order to guide a tattooing robotic arm (with rotationaland translational degrees of freedom) on the tattoo zone. A digital 3Drepresentation of the surface of the body part (or the local vicinity ofthe tattoo area) may be constructed, such that relative location andorientation of the skin surface may be calculated with respect to thelocation and orientation of the tattooing head on the robotic arm. This3D representation may be constructed by comparing images from multiplecameras (e.g., passive stereo vision), by projecting fiducial light orlaser on the surface with a known pattern and detecting thecorrespondence of the pattern on the camera image to extract the depthinformation (active stereo vision), or by a combination of multiplecameras and pattern-projection in a hybrid approach. The machine visiondevice (or other optical analysis and machine vision systems disclosedherein) can include such multiple cameras, as discussed in connectionwith FIG. 21.

A 3D surface representation may also be constructed using point-wisedistance measurement and mapping systems, such as LiDAR. The 3D pointcloud collected from such systems is then used to construct a continuousmodel of the skin surface. The digital 3D representation of the skinsurface may help actuating the automated machine to position the machinetattoo head in the vicinity of the tattoo zone and approach it with theproper angle (both the angle of the head and angle of approach), thatis, close to normal or with an appropriate angle to the normal to theskin surface in the vicinity of the dot to be tattooed. This may be ofinterest when the body part is not held in a plane or displays a complexgeometry in which the tattoo area may not be generally flat or cannot beflattened. A contactor may not be used for the flattening of the skin ifsuch 3D model of the body part is generated that clearly identifies thenormal to the skin in the vicinity of all the positions to be tattooed.However, our preferred embodiment may include a contactor in order toincrease the stability of the skin where the tattoo is to be performedand to increase the positioning resolution. A ranging mechanism, whichmay be contact-less or involves contact, such as laser, ultrasound orfeeler rangefinders may further be used to identify the skin to needletip distance with high accuracy (less than 50 μm, 75 μm, 100 μm). Themeasured tattoo-head-to-skin distance may be used in combination withthe puncture setting from the dot parameter table (i.e., needleextension measured beyond surface of the skin), to calculate the totalextension of the needle that will deposit ink at the correct depth inthe skin. The machine vision device disclosed herein (or other opticalanalysis and machine vision systems disclosed herein) can include LiDARsensors, multiple cameras, light emitter (e.g., lasers), scanners,projectors, etc.

A mapping method, such as the stencil-based machine vision technologyexplained in FIGS. 5A-6B and 11A-11D, can be used (i) to map dots on thereference tattoo design to points on the skin surface and (ii) toaccount for any deformation of the skin if the skin is not tattooed inits relaxed configuration, and (iii) to achieve a high tattooingresolution and relative spatial accuracy in the order of ˜50 um.High-accuracy mapping may be also be achieved based on other embodimentsof the machine vision methodology, for example, rather than a stencil,based on the detection of (i) skin natural fiducials (such as moles,creases, folds, hair roots, etc.) and other naturally present dermalfeatures or other sub-dermal features such as blood vessel networkswhich may be visible in some parts of the electromagnetic (EM) spectrum,or (ii) other synthetic fiducials transferred to the skin or projectinga pattern of light or laser on the skin to serve as fiducials. In thesame manner as in FIGS. 5A-6B and 11A-11D, these fiducials may be usedby the machine vision algorithm to identify the target locations (e.g.,target puncture locations) on the skin while compensating for skindeformation. The reference state of the skin fiducials which is used inthe deformation algorithm may be collected by an initial scan of thebody part in a relaxed state, performed before the tattooing operationbegins. The machine vision technology can be used to determinerelationships between changes to the skin and used to select one or morepuncture sites based on the determined relationships to reduce dotplacement deviation when the skin is in a natural state.

The 3D model of the skin surface (the global geometry of the body part)may also be used for this mapping in certain cases, however, it may notprovide high positional accuracy, especially on nearly-flat or smoothsurfaces unless used in combination with skin fiducials as describedabove. One embodiment of a hybrid localization and mapping of the tattooarea may be performed using other depth finding or 3D topology/surfacereconstruction methods such as lidar, laser, ultrasound ranging or anyother technique that may grossly identify the location of the body partas well as generate a three-dimensional, dynamically updating model ofthe skin surface. For instance, a single camera may be used to map thebody part in three dimensions by adding a projected grid on the bodypart using a projection apparatus. The movement of the dynamicallyupdating model of the skin and body part may be used to provide anadditional layer of safety by identifying when the tattoo area isshifted away from the needle tip. The needle may then be retracted andthe machine may reposition itself to realign the tattoo mechanism withthe tattoo area to resume tattooing. The skin may also be flattenedlocally, such in the use of a contactor, and the local vicinity of thetattoo area only evaluated on a flattened area. Accordingly, 3D modelingof the skin surface can be used to compensate for skin changes.

The stencil-based machine vision technology described in FIGS. 5, 6, 7may be used within an augmented reality (AR) framework, which allows thecustomer to view, dynamically-modify and confirm the placement of thetattoo design on their body. In some embodiments, the outline of thetattoo design (e.g., design 340 a in FIG. 5A) is not included in thetransferred stencil, and instead AR will be used to virtually overlaythe tattoo design on the body part where the stencil is applied. In oneembodiment of this AR framework, a high-resolution camera is used toview the body part and its video stream is processed by a computer (orother processing device) where the digital tattoo file is loaded. Thecomputer or controller performs the machine vision algorithm (e.g.,algorithm described in FIG. 7), and upon detection of the stencil in theimage(s), it is able to map the dots in the tattoo design onto thestreaming camera image(s), while accounting for any deformation orcurvature of the body part. A high-resolution, realistic render(simulation) of the tattoo design, which may represent the differentcolors in the tattoo design, account for the appearance of diffusion andfading of ink over time, etc. is generated and overlaid on the cameraimage using this mapping. The processed stream of image(s) can bedisplayed on a large screen or display for viewing by the client, wherethe images may be mirrored (flipped horizontally) to mimic the feel of amirror. The client may be provided with a controller (for example, atouch-screen interface, mouse, or other inputs discussed in connectionwith FIG. 20) which feeds into the machine vision (MV) software todynamically-manipulate the placement of the tattoo design with respectto its original location on the reference stencil. The modificationsmade by the client are processed by the MV algorithm during the render,and reflected on the screen in real-time. This manipulation may allowfor rotation and translation, and in some cases scaling of the tattoodesign, with the requirement that the entire design remains within thebounding box of the stencil fiducials (341 a in FIG. 5A). After theclient decides on a placement of the tattoo design, they may confirmtheir choice on the interface, which may be time-stamped, digitallysaved and may constitute a digital signature. Parameters oftransformation that represents the updated position (translational androtational displacement and scaling with respect to the originalposition) are appended to the digital tattoo file, or transferred intothe automatic tattoo machine by other means. During the tattooingoperation, the transformation parameters are used to update thecoordinates of all tattoo dots in the tattoo file, resulting in thedesired placement of the tattoo on the skin. The above methodology ofpreviewing, dynamically-modifying and confirming the placement of atattoo has the following advantages over static stencil-based previewingmethods: (i) the client is able to modify the placement of the tattoodesign to their preference, (ii) the AR-based method potentially savestime by avoiding repeated stencil applications, (iii) it minimizes riskof future disputes and maximizes client satisfaction related to theoverall aesthetic look of the tattoo on the body part, as it providesrealistic simulation of the final tattoo design on the client's skin,(iv) it provides a multi-color preview of the tattoo design which is notpossible with conventional stencils used in tattooing, and (v) becausethere is no obstructing tattoo outline (340 a in FIG. 5A) on thestencil, the information content of the encoding-pattern (342 a in FIG.5A) is maximized, which increases the accuracy, precision and operatingconfidence of the MV algorithm, and reduces the risk of error.

Another embodiment of the AR framework may use AR googles worn by theclient. The googles may be utilize a built-in camera, whose video streamis processed using MV algorithms, as described herein, and the design isrendered on the body part based on the dynamically-chosen parameters ofplacement. In this embodiment, the processed images with the rendereddesign can be fed into the googles for an AR experience, which displaysthe tattoo on the client's body. In some embodiments, simulated imagesof the tattoo design viewed by the client. The simulated images of thetattoo design can be overlaid on the identified target site in images ofa site. Simulated images of the target site showing the tattoo design onthe subject's body part can be viewed by the subject via a display, ARgoogles, display mechanism, computer, mobile device, or another viewingdevice. In some embodiments, a rendering of a tattoo design is mappedand projected (e.g., via a light-based projector) on the skin. This canprovide pre-visualization of the design with or without application ofany stencil. If a stencil is applied, the simulated images can be keyedand positioned with the applied stencil. The system can receive userinput via a user interface and generate operations to modify a tattoodesign based on the user input, and the system can modify the appearanceof the tattoo design based on the simulated images of the target site.The modification of the tattoo design can include translating, resizing,rotating, stretching, cropping, adjusting the color, etc. Thepre-visualization can be performed prior to visiting a studio or retaillocation and/or at the studio or retail location using, for example, amirror-LCD, AR googles, an LCD monitor, a mobile device, alight-projection-based system, etc. Visualization can also be performedduring the tattoo process to visualize section(s) of the tattoo to beapplied.

Another embodiment of the AR framework may use projection of light tosimulate the tattoo design directly on the client's skin, rather thandisplaying the render on a screen. In this embodiment, the camera isused to collect images of the body part and the MV algorithm maps thedesign with the appropriate deformation to comply with the body part, asexplained before. A render of the design is fed into a light-projectingdevice, which is placed very close to the camera, and facing the samedirection. The focal length of the projector could be automaticallyadjusted by the MV algorithm, by comparing the detected size of thestencil on the camera image to the reference stencil, and calculating anapproximate distance to the body part based on this comparison. Anydifferences in the axes of view of the camera and the projector, may beaccounted for when constructing the rendered image, to project thedesign with the right direction, orientation and scaling on the client'sbody. In some embodiments, multiple AR frameworks can be used. Forexample, the machine vision system can analyze a body part or targetsite and determine which air framework may provide the optimal clientexperience. The AR output can be compared to reference AR output toconfirm visual accuracy. AR components can communicate with controllersdisclosed herein via one or more wireless connections (e.g., via aBluetooth connection, local Wi-Fi connection, local area network, etc.),wire connections, or the like. In some embodiments, simulated images ofthe tattoo design can be generated. The simulated images of the tattoodesign can be overlaid on the identified target site in images of asite. Simulated images of the target site showing the tattoo design onthe subject's body part be viewed by the subject via a display, ARgoogles, display mechanism, computer, mobile device, or another viewingdevice. This can provide pre-visualization of the design with or withoutapplication of any stencil. If a stencil is applied, the simulatedimages can be keyed and positioned with the applied stencil. The systemcan receive user input via a user interface and generating operations tomodify a tattoo design based on the user input and can modifying theappearance of the tattoo design on the simulated images of the targetsite. The modification of the tattoo design can include translating,resizing, rotating, stretching, cropping, adjusting the color, etc.

Tattooing Apparatus with Frame and Contactor

FIG. 8 is an isometric view of another embodiment of a tattooingapparatus 400. The description regarding tattoo apparatus 100 cangenerally apply to tattooing apparatus 400 as well. For example, in oneembodiment, tattooing apparatus 400 may comprise of a tattoo frame 403,a tattoo shuttle 404, an arm 410, a contactor 420, a machine visiondevice 430, a needle structure 440, and a suction system 450.Additionally, tattooing apparatus 400 may comprise more or fewer othercomponents including but not limited to one or more motors, one or moreactuators, one or more controllers, one or more gantries, a restsurface, a cantilevered tattoo machine, a needle cartridge, a needle, aplunger, a spring, a piston, a cam, an imaging device, a lens, etc. Insome embodiments, the tattooing apparatus 400 can include one or morecontrollers, motors (e.g., drive motors, stepper motors, etc.), gantrydevices, linear slides, rails, sensors (e.g., position sensors,accelerometers, etc.), or the like. In some embodiments, tattooingapparatus 400 can work with one or more separate controllers to form atattooing system. Tattooing apparatus 400 can be configured based ondesired characteristics for a particular tattooing process and mayutilize any and all methods and/or components consistent with thepresent disclosure.

Marketplaces, Tattoo Selection, and Application

FIG. 9 is a block diagram of an embodiment of a simplified tattooselection and application process 500. A subject may browse and/orselect a design 510 from an online tattoo marketplace. In oneembodiment, the subject may browse and/or select a design 510 from theonline tattoo marketplace using a device 509 (or controller 108 of FIG.1B), which may be, for example, a personal handheld device, smartphone,or a computer. The browsing and selection 510 may be done via a mobileapp and/or website that allows access to the online tattoo marketplace,through which subjects may perform actions including, but not limitedto, browsing, selecting, saving, rating designs, uploading, creating aprofile, booking appointments, participating in auctions, or buying. Inone embodiment, the online tattoo marketplace may be a global onlinetattoo marketplace where artists or users may upload, license, and/orsell their designs irrespective of their physical location. Artists maybe paid a royalty based on selection and/or licensing of their designsby subjects through the app and/or website. Browsing, selection, andpayment process may vary by location, as well as, by individual artist.

After browsing and selection 510, the subject may preview the selecteddesign via augmented reality. In one embodiment, the subject may previewthe selected design 520 via the device 509 using augmented reality tovisualize how the design may look on a desired area to be tattooed.Augmented reality is a form of reality that has been adjusted in somemanner before presentation to a user, which may include, e.g., virtualreality (VR), passthrough augmented reality (AR), mixed reality (MR),hybrid reality, or some combination and/or derivatives thereof.Augmented reality content may include completely generated content orgenerated content combined with captured content (e.g., real-worldphotographs). The augmented reality system that provides the augmentedreality content may be implemented on various platforms, including ahead-mounted display (HMD) connected to a host computer system, astandalone HMD, a mobile device or computing system, a projectionsystem, or any other hardware platform capable of providing artificialreality content to one or more viewers. For example, a tablet or mobilephone with a camera on the back can capture images of the real world andthen display the images on the screen on the opposite side of the devicefrom the camera. The device can process and adjust or “augment” theimages as they pass through the system, such as by adding tattoodesigns. In some implementations, a similar process can be performedusing a virtual reality or mixed reality headset, which allows lightfrom the real world to pass through a waveguide that simultaneouslyemits light from a projector in the mixed reality headset, allowing themixed reality headset to present virtual objects intermixed with thereal objects the user can see. Previewing 520 may be available, forexample, prior to and/or after selection of the design. Followingpreviewing step 520, a subject may apply the design to the desired areausing any of the systems or methods described. In one embodiment, asubject may undergo the tattooing process 300 using tattoo apparatus 100or tattooing apparatus 400. In other embodiments, other embodiments ofthe systems and methods consistent with this disclosure may becontemplated to apply the tattoo. Application may involve utilizing astencil 530, which may be any stencil compatible with the systems andmethods described herein. Stencil application is discussed in connectionFIGS. 5A-6B and stencil rejection and approval are discussed inconnection with FIGS. 11A and 11B.

With continued reference to FIG. 9, the precision tattooing may beperformed 540 using any system or method disclosed or consistent withthis disclosure. In one embodiment, the tattooing may utilize machinevision. Following tattooing step 540, an aftercare treatment may beapplied 550. In one embodiment, aftercare treatment 550 may promote, forexample, healing, coloring, and/or fixation. Following the aftercaretreatment 550, the tattoo may be revealed in step 560. The tattoo canbe, for example, a micro tattoo, blackwork tattoo, color tattoo, realismtattoo, fine-line tattoo, or the like.

In some embodiments, a tattoo system receives a user's selection of atattoo design and sends authorization data for the user's selection. Theautomatic tattooing apparatus can use the authorization data todetermine whether to robotically apply a tattoo. The authorization datais sent to the user's device 509, the automatic tattooing apparatus, orboth. The authorization data can include a token or credit for applyingthe selected tattoo design. The mobile application can manage an onlinetattoo marketplace that allows browsing of tattoo designs and selectingof the tattoo design. The application process 500 can include otherfeatures, steps, and processes disclosed herein.

Needle Cartridges

Different types of components can be incorporated into tattooingsystems. FIG. 10A is an isometric view of an embodiment of a needlecartridge 641 suitable for system 76 of FIG. 1A, system 90 of FIG. 1Band system 400 of FIG. 8. FIG. 10B is an exploded view of the needlecartridge 641 of FIG. 10A in accordance with an embodiment of thedisclosure. Referring to FIGS. 10A and 10B, the needle cartridge 641 maybe removably coupled to the tattooing apparatus (e.g., apparatus 100 ofFIG. 1B) such that switching between needle cartridges 641 may occureasily. The needle cartridge 641 may be a disposable component holding atattoo needle and composed of the needle spring 644, plunger 645, andone or more needles 642. In other embodiments, however, the needlecartridge 641 may be suitable for refill, reuse, or limited use, and maycomprise more or fewer than the parts mentioned. The needle cartridge641 may contain at least one sensor 646 such as, for example, anelectrical wire for galvanic sensing. In one embodiment, the needlecartridge 641 can comprise of a wire coupled to a non-puncturing end 647of the needle 642 and is configured to act as a galvanic electrode thatcan begin measuring or exciting the skin as soon as the needle 642contacts the skin of the subject. The sensor 646 does not need to be awire, however, and can be any type of sensor suitable for the desiredconfiguration.

The needle cartridge 641 may also comprise of a housing 650, which maycontain, for example, a cartridge tip 651, cartridge body 652, andcartridge cap 653. The housing 650 may vary in components, size, shape,design, color, and material depending on the desired configuration ofthe needle cartridge 641 to be used for tattooing. In some embodiments,the sensor 646 may extend through an ink inspection hole 654 in thecartridge body 652 for coupling to the tattooing apparatus 100. The inkinspection hole 654 may, for example, facilitate inspection of properink quality and distribution. In other embodiments, the sensor 646 doesnot extend through the ink inspection hole 654 and/or from othercomponents of the housing 650 and may be located or coupled elsewhere.The ink inspection hole 654 may be sealed or opened to ambient air andmay be connected to a fluidics system for the delivery of ink or otherfluids to the cartridge 641.

The needle cartridge 641 can vary in the number, size, shape, type,sharpness, and arrangement of needles 642. In one embodiment, forexample, needle cartridge 641 may utilize 3RL type needles in a slightlystaggered arrangement. In other embodiments, the size, grouping, numberof needles in the grouping, and arrangement may vary depending on thedesired configuration and design. For example, any size greater than orless than 3 (e.g., size 2, 5, 7, 10, 12, etc.) needles can be used. Inanother example, any grouping type, needle gauge or taper may be used(e.g., RL, RLXT, RLXP, RS*T, F, M1, M2, M1C, etc.) along with any numberof needles in the grouping. The needle type, shape, number, size,grouping, number in grouping, arrangement, etc. can be selected based onthe use with the systems and methods of the present disclosure. Theneedle cartridge 641 may be any of a variety of types of cartridges,including but not limited to, custom cartridges, third party cartridges,generally available cartridges, or any other cartridge capable ofoperation with the systems and methods of the present disclosure.

The ink cartridge (not shown) may be installed as a removable componentto the tattooing apparatus and/or needle cartridge 641 such thatswitching between ink cartridges may occur easily. The ink cartridge canbe configured to allocate ink via capillary action, tubing, and/or pumpsat prescribed intervals from an anti-cross-contamination ink supply. Inone embodiment, the ink cartridge may be a single use, disposablesterile ink cartridge with sufficient ink for a tattoo. The inkcartridges may contain different amounts and/or colors (e.g., black,red, blue, green, skin tone brown, etc.) and/or types of ink. Theoperator may choose one or more suitable ink cartridges depending on thetattoo. In some tattooing processes, one or more of the same ordifferent ink cartridges may be used. In some embodiments, the inkcartridge may be suitable for refill or reuse. In one embodiment, thetattooing apparatus 100 and/or needle cartridge 641 may be configured tocouple with multiple ink cartridges simultaneously. The ink cartridgetype, shape, number, size, color, arrangement, etc. may vary so long asthe ink cartridge is capable of use with the systems and methods of thepresent disclosure. The ink cartridge may be any of a variety of typesof cartridges, including but not limited to, custom cartridges, thirdparty cartridges, generally available cartridges, or any other cartridgecapable of operation with the systems and methods of the disclosure. Insome embodiments, the ink cartridge may be a component of the needlecartridge 641 or may be a separate component that may be coupled to theneedle cartridge 641 and/or tattooing apparatus 100. The arrangement andconfiguration of the ink cartridge may vary depending on the desiredconfiguration of the needle cartridge 641 and/or tattooing apparatus.The ink cartridge may be anything capable of holding and distributingink such as, for example, an ink reservoir, ink pack, or the like.

Stenciling, Skin Analysis, and Related Technologies

FIGS. 11A and 11B are illustrative diagrams of an example rejection andexample approval of stencil positioning, respectively, in accordancewith another embodiment of the disclosure. In one embodiment, machinevision device (e.g., machine vision device 130 of FIG. 2) may assessstencil 710 positioning via an interrogation window 720. In oneembodiment, interrogation window 720 may span a portion of the area tobe tattooed. For example, in one embodiment, the interrogation window720 can correspond to dimensions of the window of the contactor, such ascontactor 120 of FIG. 2. With continued reference to FIG. 7A,interrogation window 720 may coincide with a field of vision of themachine vision device and may be configured to span the entirety of thetattoo field. In one embodiment, the interrogation window 720 maytransition between multiple positions 722 of a scan zone 721 to assesspositioning of the stencil 710. For example, the interrogation window720 may be configured to detect a presence and/or absence of one or morereference features 730 and/or one or more tattoo dot locations 740.Approval of the positioning of the stencil 710 may then be based atleast in part on the detected presence and/or absence of referencefeatures 730 and/or tattoo dot locations 740. In one embodiment, themachine vision device 130 can utilize a machine vision algorithm tocompare the detected reference features 730 and/or tattoo dot locations740 with a vector-based stencil drawing. In other embodiments themachine vision algorithm may be performed by a controller such ascontroller 109.

FIG. 11A is an illustration of an example rejection of stencilpositioning in accordance with one embodiment. For example, stencil 710can be rejected as a result an absence of reference features 730 of thestencil 710 anticipated at particular locations within the interrogationwindow 720. Alternatively, stencil 710 can be rejected as a result anabsence of a sufficient number of references features 730 near thetargeted tattoo dot locations 740 anticipated at particular locationswithin the interrogation window 720. An absence of particular referencefeatures 730 within the interrogation window 720 may be determined basedat least on a comparison with reference data, such as a referencedrawing, a vector-based stencil drawing, or the like. Rejection mayoccur when there is insufficient context within the interrogation window720, suggesting reapplication of the stencil 710 may be required.

FIG. 11B is an illustration of an example approval of stencilpositioning in accordance with one embodiment. The applied stencil 710positioning may be approved based at least in part on a presence ofreference features 730 within the interrogation window 720. In oneembodiment, a presence of particular reference features 730 within theinterrogation window 720 may be determined based at least on acomparison with a vector-based stencil drawing. In one embodiment,reference features 731 may be added to and/or removed from the stencil710 to promote approval of the stencil 710 positioning. In anotherembodiment, stencil 710 repositioning and/or reapplication may be enoughto promote approval of the stencil 710 positioning. Approval may occurwhen there is sufficient context within the interrogation window 720,suggesting stencil 710 may be properly positioned and tattooing mayproceed. Once approved, ink can be deposited at the tattoo dot locations740 using reference feature-based positioning in which the referencefeatures 730 can be used as landmarks.

FIGS. 11C and 11D are simplified illustrative diagrams of an examplerejected stencil and an example approved stencil, respectively, inaccordance with one embodiment. FIG. 11C depicts in a simplified manner,an absence of reference features within a contour of the design (e.g., avirtually applied design) sufficient for approval. FIG. 11D depicts in asimplified manner, approval despite a limited absence of some referencefeatures suitable for generating targeted tattoo dot locations. Forexample, approval may still occur if the absence of reference featuresand/or tattoo dot locations is sufficiently limited and/or in anon-critical region such that the tattooing process may proceed. Themethods or steps discussed in connection with FIG. 11A-11D can be usedfor skin tracking, skin analysis, deformation compensation, and othertechniques disclosed herein, such as the techniques discussed inconnection with FIGS. 24A-24C.

FIG. 12A is an illustrative diagram of an example of puncture events inaccordance with an embodiment of the disclosure. The needle contacts theexposed skin surface and begins to puncture the epidermis. The maximumextension corresponds to the needle being located at the ink depositposition. After depositing ink, the needle is pulled out of the skin.The depth of ink deposition can be determined based on the position atcontact, the position at initial puncture, the position at maximumextension, and/or the angle of attack of the needle, as discussed inconnection with step 310 of FIGS. 3A and 3B.

FIG. 12B is an illustrative conductivity signal of one embodimentutilizing a galvanic sensor and its first derivative with respect totime for a single puncture. Additionally, the needle extension may beobtained using the measurement from an angular encoder (e.g., an angularencoder on the needle motor 111 of FIG. 2) or other sensor. In general,the puncture events 1-4 are detectable by determining the extrema ofthese three illustrated curves. Puncture event 1 corresponds to initialcontact. Puncture event 2 corresponds to puncture initiation. Punctureevent 3 corresponds to ultimate or maximum depth needle position. Postpuncture area 4 corresponds to post puncture needle exit. Examplepuncture events are described in detail below.

The puncture events in accordance with an embodiment are identifiedusing an algorithm identifying the local extrema in the conductivitysignal during puncture and the local extrema in the first temporalderivative of the conductivity signal during puncture. The initialcontact is identified when conductivity becomes non-zero and the firsttemporal derivative of the conductivity signal becomes non-zero. This isbecause the needle closes the electrical circuit formed by the skin andthe electrodes when the needle is in contact with the skin. The positionat initial puncture is identified by a trend artifact in the firsttemporal derivative of the conductivity signal following initialpuncture. This change in trend is observed in the first temporalderivative of the conductivity signal because of the shift from surfaceconductance to subdermal conductance.

When puncture of the epidermis finally occurs, the needle becomes incontact with the inner tissue which is more conductive than the outerlayer of the epidermis, while the surface of the skin may be conductive,which results in an increase of bulk conductivity and a decrease ofsurface conductivity.

The position at maximum extension is identified by the analysis of theposition of the needle using the needle motor 111 of FIG. 2 angularencoder with a known alignment when the needle is at its deepestsetting. The signal from this encoder can be superimposed with thesignal from the galvanic response to synchronize the temporal andpositional signals. The position of the needle may be calculated usingthe equation for an eccentric cam. The post puncture event is identifiedwhen the conductivity signal and the first temporal derivative of theconductivity signal suddenly reduce. This is because the needle islosing contact with the tissue.

The conductivity signal output can be analyzed to determine additionalinformation, including the tissue characteristics (e.g., mechanicalproperties of the tissue, electrical properties of the tissue, orthickness of tissue layers), performance of the tattoo system, or thelike. The number and pattern of locations at a targeted area that areanalyzed can be selected based on the characteristics of the tattoo tobe applied at the target area. For example, the number of locations canbe increased or decreased based on how the tissue characteristics varyacross the target area.

The techniques discussed in connection with FIGS. 12A and 12B can beused to determine first contact, puncture initiation, needle position,needle exit, and other puncture events based on data from other sensorsdisclosed herein. The puncture force used to drive a needle through eachlayer of skin can be monitored to identify such events. This is becauseeach layer has different mechanical characteristics that can beidentified using analytics and Al based algorithms disclosed herein.Puncture force versus displacement curves can be generated to identifythe events of interest. Optical sensors, pressure sensors, logicalsensors, or combinations thereof can be used with the puncture forcedata to identify the events. In other embodiments, sensors cannoninvasively analyze sites along the skin to detect skincharacteristics. The sensors can be ultrasound sensors, optical sensors(e.g., near infrared sensors, infrared sensors, etc.), acoustic sensors,or the like. In some embodiments, both noninvasive and invasivetechniques are used to analyze the skin. Results from both techniquescan be compared and used to generate predictive skin thicknesses atvarious locations along the site.

FIG. 13 is a flow chart of a method 808 for acquisition and usage of atraining set for the development, experimental calibration, and/oroperation of a method for sensor-based control of needle extensionduring tattoo ink deposition. In step 810, parameter space is identifiedby, for example, identifying internal and external variables orproperties which may influence the visual outcome of a tattoo dot. Theinternal variables can include: (i) needle extension, (ii) number ofpunctures per location, (iii) type and geometry of the needle, and/or(iv) type of the ink. Among these, (i) and (ii), are the puncturesettings, which may be controlled by an algorithm based on sensor dataduring tattooing operation. The external variables or properties caninclude: geometric properties of the skin, such as thickness of itsvarious layers and physical properties of the skin, such as massdensity, water content, presence and properties of the supporting(backing) tissue, such as bone or muscle, age, race and gender of theperson, (viii) body mass index, and/or and body hydration. The variablesfor the parameter space can be selected based on the functionality ofthe tattooing system, tattoo to be applied, or the like.

In step 811, data 816 can be collected by, for example, performing oneor more experiments on a set of participants to identify the internaland external variables that influence the puncture sensing and inkdeposition processes, and to find the relations or correlations betweenthese variables. The participants and internal puncture settings can bechosen to increase or maximize the range of internal and externalvariables. The experiments can involve administration of wet (inked)punctures on the skin as well as dry punctures in the vicinity of thewet punctures. The experiments are performed and the collected data canbe organized in a dataset as: the puncture settings (i, ii), needle andink type (iii, iv), meta information about the participants related toexternal variables (v-ix); sensor data from the dry puncture experiments(x), and high-definition pictures of the resulting inked dots (xi), andtheir numerical scores (xii) which can be calculated in Step 812. Theimages (e.g., pictures) of tattoo dots (xi) may be appended withadditional images taken at subsequent stages of the skin's healingprocess. In Step 812, inked dots can be assigned numerical scores (xii)based on their aesthetic quality, which can then be used to train andvalidate the puncture control method. Aesthetic quality can be inputtedby a user or determined using an automated scoring protocol. Forexample, scores may be manually (by human) or automatically (by imageanalysis) assigned, based on the following visual aspects: (i) diameterof the dot in comparison to the expected diameter, (ii) circularity ofthe dot, (iii) sharpness of the edges, or the degree of diffusion,and/or (iv) presence of blowout, or another undesirable outcome. Next, amodel can be trained from the collected data to predict the dot outcomeas a function of one or more puncture parameters and otherinternal/external variables. The model can be configured to determine orselecting needle extensions, number of punctures, needle tipconfiguration, and/or number of needles so as to affect tattoo dot size,aesthetic quality, color saturation, color gradient, and/or color tone.

Two alternative models are described in steps 813 and 814. In Step 813,a mechanistic model of the needle and the skin may be developed whileaccounting for the uncertainties in the data and input/outputquantities. The input of the model may be based on data collected fromone or more sensors, such as galvanic sensor data which correlates withthe contact of the needle and the layers of the skin, load-cell datawhich correlates with the force on the needle or motor encoder datawhich correlates with the angular position or angular velocity of theneedle. The times of individual puncture events, such as (i) firstcontact with skin (event 1 in FIGS. 12A-B), (ii) puncture events ofindividual skin layers (event 2 in FIGS. 12A-B), and (iii) needle exitfrom the skin (event 4 in FIGS. 12A-B), are detected from the galvanicsensor data as shown in FIG. 12B. The motor encoder data in combinationwith the detected times of puncture events (i) and (iii) may be used tocalibrate a model of the oscillatory needle extension as a function oftime. This model can be constructed by calculating the conversion of theeccentric rotation of the cam to the linear extension of the needle.This model may be used to predict the time of maximum needle extension(iv) (event 3 in FIGS. 12A-B), and the extension of the needle at thetime of each puncture event (i-iii) into position of the needle tip. Thedistance traveled by the needle between individual events (i-iv)constitute intermediary output variables of the model, which may providevaluable information pertaining to the tattoo ink delivery. For example,distance traveled between first contact (i) and first puncture (ii) is(a) the initial deflection. The distance traveled between the times ofpuncture of individual layers (ii) may be correlated to the (b)thickness of these layers. The distance traveled between first contact(i) and maximum needle extension (iv) is (c) the max extension. Thedistance between max extension (iv) and needle exit from skin (iii) is(d) the posterior max extension. A combination of the intermediaryoutput variables (a,b,c,d) may be statistically related to thepenetration depth of the needle, the depth of ink deposition, or thetattoo dot quality (xii) in the experimental dataset. The statisticalrelation between the intermediary output variables (a, b, c, d) and thetattoo dot quality (xii) may be identified using a partial least-squaresregression (PLS) method, or other regression methods such as standard orconstrained least square approaches, regularized minimizations,principal component analysis or other. The goodness of fit, correlationcoefficients, or other statistical measures of the model precision withrespect to the predicted output, may be calculated from the dataset toprovide a confidence index to the predictions during tattooingoperation.

As an alternative to step 813, a machine-learning method, such as aneural network, could be trained in step 814 from the collected dataset,in order to predict the correct puncture settings to achieve ahigh-quality dot as a function of the sensor data and any availablemeta-information related to external variables (v-ix). In this approach,the dot score is used as the objective set and the sensors signals arethe input and the puncture settings are the searched variables. Thepuncture events may or may not be defined as intermediary input andcollected dataset may be enhanced until a statically representativedataset is obtained.

In step 815, the model developed in step 813 or 814 is used to predictthe correct puncture settings which would create a high-quality tattoodot, for each location where dry punctures are applied. The predictedpuncture settings are appended to a parameter table 316 of FIGS. 3B and3C.

Shading can be obtained by, for example, varying the needle extension,number of punctures (e.g., number of punctures at a certain position orarea), amount of applied ink (e.g., volume of ink applied for eachpuncture event or set of puncture events), or combinations thereof. Forexample, shading can be achieved by varying the amount of ink that isdelivered at a certain dot by depositing ink at a different depth withinthe skin, or by varying the number of punctures at the same position,etc. In some embodiments, a dot with more punctures will receive moreink than a dot with fewer punctures. Similarly, a dot created by shallowpunctures will preserve less visible ink after healing than a dotcreated by deeper punctures, provided that the deeper punctures are notso deep as to result in a defective dot, for example in the case of ablowout, where ink is dispersed due to diffusion and immune response. Insome instances, a deeper ink deposition can result in a more diffuseddot of a lighter shade. These techniques can be used to create differentshades of color. For example, an area with fewer punctures per dot cancreate a lighter shade than an area with more punctures per dot.Similarly, an area of the tattoo can be performed with shallowerpunctures to create a lighter shade than an area where deeper punctureswere performed. In some procedures, both depth and puncture number perdot can be selected to achieve various shades or to compensate for otherconstraints in the tattooing process.

Another process for shading includes varying the spatial density ofdots. For example, closely spaced dots can result in a darker shade thanwidely spaced dots. The pattern, pitch, and/or spacing of dots can bedetermined based on the desired shade of the tattoo. Dithering, inkdeposition depth, and/or number of punctures per dot may be usedtogether or individually to realize various shading in a tattoo. Duringtattooing operation, the puncture settings (e.g., needle extension,number of punctures, ink delivery rates, target depth, etc.) for atattoo dot may be determined based on the target characteristic(s)(e.g., shade, color, etc.) of the dot in the artwork, which may be savedin a digital tattoo file (e.g., metadata 1217 in FIG. 19), and a methodof predicting the puncture settings to achieve the desired shade, coloror ink saturation, such as the method 808 of FIG. 13.

In some embodiments, predictions are generated using the analyticsprediction methods of step 813 and AI-based predictive methods of step814. The method 808 can include using output from both steps 813, 814 todetermine predictions, e.g., by using confidence factors determined forone or both processes to weight a combination of the results or toselect which output to use at any given time. For example, the output ofthe machine-learning method can include a value in a range for apuncture setting, where a difference between the value produced at thenearest of the range can be the confidence factor, and the results ofthe machine-learning method are only used when that confidence factor isabove a threshold, otherwise the analytics-based method is used.Alternatively, the method 808 can select one of the outputs from step813 or step 814 as the prediction. The selection can be based onanalysis of the collected data and historical prediction accuracy forsimilar data. The model from steps 813 and/or 814 are used to predictthe correct puncture settings.

A robotic tattooing system can include a portable automated handheldtattoo device. The handheld tattoo device can be conveniently carried bya user and applied to a subject. This allows tattoos to be applied at awide range of settings, including at tattoo studios, spas, homesettings, or the like. During a tattooing session, the handheld tattoodevice can be manually repositioned (e.g., manually carried and placed)at desired locations.

Handheld Tattoo Device and Related Technology

FIG. 14 is an isometric view of a handheld tattoo device or unit 900(“handheld tattoo device 900”) in accordance with an embodiment of thetechnology. The handheld tattoo device 900 can include a grip or handle902, a main body 904, and a needle assembly 905. A control element 901can be used to control operation and can include a trigger, a pushbutton, a switch, a finger interface, and/or an actuatable element. Thecontrol element 901 can be used to, for example, control operation ofthe needle assembly/actuator, start/stop a tattoo protocol, or the like.A sensor 903 can include one or more indicators, levels, accelerometers,gyroscopes, etc. In some embodiments, the sensor 903 is an indicator(e.g., one or more bubble levels) used to set or correct (1) relativepositioning of the tattoo device 900 with respect to the skin surface,(2) the orientation (e.g., horizontal orientation or verticalorientation) of the tattoo device 900, or (3) the angular position ofthe tattoo device 900 relative to a reference plane.

FIG. 15 illustrates a handheld tattooing system 908 in accordance withan embodiment of the technology. The handheld system 908 can include thehandheld tattoo device 900, a computer and/or controller unit 913 (e.g.,a PC/tablet running the control software), controller unit 912, a safetydevice 914 (e.g., pedal safety switch), and an electrode assembly 915(e.g., skin electrodes). The components can be integrated into orcoupled to the handheld tattoo device 900. For example, the componentscan be detachably coupled to the handheld tattoo device 900. Thehandheld tattoo device 900 can include one or more rechargeable powersources. In some embodiments, the handheld tattoo device can be poweredvia an external power source. The handheld tattooing system 908 can beincorporated into systems with features and functionality discussed inconnection with FIGS. 1 and 21.

FIG. 16 is an exploded view of components of the handheld device 900 inaccordance with an embodiment of the technology. The handheld device 900can include a contactor 1004 (or contactor 1106 in FIG. 18C) with one ormore windows or openings to facilitate visualization of the skin. Thecontactor 1004 can be configured to keep a region of the skin surfacegeneral flat. In some embodiments, the contactor 1004 has a generallycircular shape, partially spherical shape, or cylindrical shape sinceonly one dot or a limited number of dots may be applied at a time. Inmulti-needle assembly embodiments, the configuration and number ofcontactors and number of openings can be selected based on whether inkis applied simultaneously or sequentially by different needlesassemblies.

The device 900 can also include an integrated lighting system thatoutputs light to facilitate operator vision of the tattoo window andopenings of the contactor. In FIG. 18C, the integrated lighting can beintegrated in the contactor holder 1006. In some embodiments, thelighting system is installed as a removable component of the device 900to allow repositioning of the lighting system. All or some of thecomponents of the handheld device 900 can be encased or include aprotective housing to protect the mechanism from being exposed to thepatient and/or operator. A removable electrical assembly can include oneor more wire harnesses used to couple the tattoo device to components,such as the control and sensing electronic box. In some embodiments, thedevice 900 has an integrated control and sensing electronic box, aninternal power source, or the like. A pedal (e.g., a safety device 914in FIG. 15) can be used to trigger off the actuator in case ofmalfunction.

Referring to FIG. 16, a load cell 1008 can be coupled to the contactorholder 1006 and to the body of the handheld tattoo device 900 in orderto sense the applied pressure from the skin to the contactor 1004. Thetattoo device 900 can also include one or more actuator mechanismssimilar to or the same as the actuator mechanism described in connectionwith arm 110 and FIG. 2.

The needle assembly 905 can be inserted into an opening 1007 of the mainbody 904. One embodiment of the disposable needle cartridge is anintegrated ink and needle disposable (or not disposable) cartridge. Sucha system includes an ink reservoir 1002 holding ink that be injected inthe needle well 1005 by pressing a piston 1001. A line or tube 1003 canbe used to deliver the ink from the ink reservoir 1002 to the needlewell 1005. Referring now to FIGS. 10A, 10B, 14, and 16, the needlecartridge of FIG. 16 and the needle cartridge 641 of FIGS. 10A and 10Bcan include one or more electrodes, electrode wires, sensors, or thelike. For example, the cartridge 641 of FIGS. 10A and 10B can containthe electrode wire 646 that provides the needle electrode connection 647to the rest of the system or device. With continued reference to FIGS.10A and 10B, the needle cartridge 641 also contains a plastic piston 645connected to the needle 642 to provide Z movement and an elastic skirt644 that provides a seal to ink and air from the ink well as well as aspring like action on the plastic piston. When assembled, air and inkcan only be released through the needle opening. The ink cartridge ofFIG. 16 can include similar components and operate in a similar manner.

The ink and needle cartridges can integrate with the needle and the inknecessary to provide desired tattooing action. When inkless measurementwith the needle is performed (e.g., dry puncture), the ink reservoirpiston is not pressed and no ink is present in the ink well. The needleis therefore operated without ink on its surface, resulting in apuncture with no ink. When performing a tattoo with ink, the inkreservoir piston is pressed in initially until ink fills up the inkwell. The ink in the ink well does not drain excessively from the needleaperture when the needle actuation is not performed due to the surfacetension of the ink. In operation, the needle actuation moves ink fromthe ink well by coating the needle surface with ink, which allows ink tobe transferred to the skin and in the skin. As the tattoo progresses,the piston of the ink reservoir is further pressed to compensate for theconsumed ink as part of the tattooing process such that the ink well isalways sufficiently full for tattooing. This can be realized by eithersensing the ink well content or by adding a prescribed amount of ink forevery, or some number of actuations of the needle. The piston of the inkreservoir can be, in one embodiment, pressed by an automated actuator,or, in another embodiment, by the operator in case of a manuallyoperated machine.

The ink and needle cartridge system may, in some embodiments, notcontain a piston to transfer ink from the reservoir to the ink well, butany other suitable mean of transferring ink from one to the other, suchas a pump, capillary action, pressure differential, or piezoelectricaction. The ink and needle cartridge may include multiple sub elements,each of which may or may not be disposable. The ink and needle cartridgemay contain sensors and electronic components to detect ink level and toauthenticate originality, quality or first usage of the ink and needlecartridge, to avoid reuse of components (e.g., inkcartridges/assemblies), quality of product and one status of product.

The handheld manually operated tattooing process can include one or moresteps discussed in connection with FIG. 3A and there can be someoptional differences in steps 302, 303, 304, 305, 306, 307, 310, 320,308. The handheld manual operation can include one or more featuresdiscussed in connection with step 310 (FIG. 3B), step 320 (FIG. 3C), andprocesses discussed in connection with FIG. 17. Referring now to FIG.3A, in a manually-operated embodiment of step 301, a portion of skinthat will receive the tattoo is prepped by, for example, shaving (ifrequired or desired), cleaning the skin surface, etc. In step 302, astencil can be applied to the portion of skin. The stencil in step 302can be similar or identical to the steps described in connection withFIGS. 18A-18C or other stenciling processes disclosed herein. Thestencil can provide information about operation of the tattooing device.The stencil can include one or more markings indicating tattooingprocedure information, specifying the order of operations for a manualoperator to execute, post tattooing session information, or combinationsthereof. In some embodiments, the stencil can provide a temporaryindication of the final design that can be reviewed and accepted by theclient. Artwork can be used to generate a stencil. An algorithm may beused to generate the stencil from the artwork. The artwork can becomprised of dots, lines, areas interpreted as dots, or the like. Thealgorithm can be selected based on the characteristics of the artwork,such as resolution of the artwork, colors of the artwork, or the like.

After applying the stencil 302 of FIG. 3A, the subject and/or operatormay check the stencil 303. The subject may review the stencilapplication and either approve or reject the design placement. Theoperator may review the stencil for quality of application. In someembodiments, for example, a stencil deposition should be such that atransfer of fiducial marks is adequate to apply a tattoo using thehandheld tattoo device without missing dots. If the stencil appearsmisplaced or the application is not accepted by client, the stencilapplication step may be repeated until accepted.

Following approval of the stencil application, lubricant can be appliedin step 304. A variety of suitable lubricants with different viscositiesand hydrophobic properties may be suitable for use. For example, alubricant with a viscosity between, for example, 10 cps and 500 cps withhydrophobic properties to increase the contact angle between inkdroplets and skin may be used. The lubricant can be chosen such that thetype and viscosity of the lubricant may allow it to protect theepidermis top surface from being stained by ink and/or increase ease ofremoval of the ink.

Step 305 may be performed after step 304. In step 305, the handheldtattoo device can be prepared to position the disposable ink deliverysystem, contactor, needle cartridge and/or protective bagging on thehandheld tattooing device. These accessories can be disposable forhygiene purposes. Electrodes (e.g., liquid electrodes) can be positionedon the skin by the operator within the vicinity of the tattooed skinarea. The electrodes can be positioned side by side or in anotherpattern, with the test electrode positioned closer to the tattoo areathan the reference electrode.

Following step 305, the tattoo device may perform a calibration routine.In step 306, the tattoo device can identify its internal zero referenceand calibrate itself. The calibration routine can include actuating oneor more actuators (e.g., actuator system) to assess correct operation inthis step. In one embodiment, calibration may comprise of using analgorithm to run a diagnostic of the sensors. Additionally oralternatively, a conductivity test may be performed to confirmconnection of one or more electrodes to the tattoo device and/or to theskin.

In the handheld operation, step 307 of FIG. 3A can be omitted, sincepositioning of the handheld tattoo device can be performed by theoperator in steps 310 and 320 for each and every dot, or a subset ofdots. In some embodiments, a tattoo process can include both step 307 ofFIG. 3A and steps 310 and 320.

At step 310, a skin puncture property acquisition can be performed. Forthe manual embodiment of FIG. 3B, the operator can identify one or moremarkers associated with a dry puncture.

FIG. 17 is a block diagram of a dry stage with dry puncture and wetstage with wet puncture in a tattoo application process step inaccordance with an embodiment of the disclosure. The robotic systems canperform the steps discussed in connection with FIG. 17 and discussed inconnection with the embodiments of FIGS. 14-16. A dry puncture is apuncture that is formed without applying ink. For example, a needle canbe inserted into the skin to form the puncture, and the needle can beremoved from the skin. The locations designated to receive dry puncturesare denoted by an unfilled feature 1101 in the example stencilillustration in FIGS. 18A to 18C. The unfilled feature configuration(e.g., shape, dimensions, diameter, etc.) can be an indicator for needleconfiguration. In one embodiment, small diameter circles indicate theuse of a needle with 3 or less tips in a tight circular configuration,and large diameter circles indicate the use of a needle with 4 or moretips in a tight circular configuration. The position can either beidentified by a numeral character (e.g., number 1104 as shown in FIG.18A) applied as part of the stencil and/or through a digitalrepresentation of the tattoo displayed on the computer or tablet thatruns the control software's graphical user interface. Other techniquescan be used to apply the numeral character.

A control module can send a command to the computing unit of the tattoosystem to configure (e.g., applying one or more settings) the tattoodevice for the skin puncture acquisition in step 1312. After a settingis applied in step 1312, the operator can position the contactor windowby centering the contactor with the marker in step 1313. FIG. 18Cillustrates positioning of the contactor window 1106 with the stencilillustration. For example, contactor can be aligned with the dry markernumbered “1”. In step 1313, the operator can also ensure that the tattoodevice is properly oriented with respect to the skin surface with orwithout using a level element, such as the level element 903 in FIG. 14.If the skin surface is oriented horizontally, the level element can bean air bubble level element used to ensure that the contactor/skinsurface are substantially parallel.

The operator can increase or decrease the pressure applied by thecontactor to the skin by applying manual force. In some procedures,belts, straps, adhesive elements (e.g., double-sided adhesive tape) areused to couple the tattoo device to the subject. The tattoo device canapply a sufficient level of force to the skin to ensure sufficientcontact between the contactor and the skin. The amount of force employedcan be measured by one or more sensors (e.g., contact sensors, pressuresensors, load cell, etc.). The amount of force can be digitally reportedby the graphical and/or sound interface of the control module. Theamount of force or pressure allowed can be between two force/pressurevalues, a lower bound optimal force/pressure and a higher, upper boundoptimal force/pressure. In some embodiments, the optimal forces arebetween 0 g and 10,000 g.

The control module can block the actuation of the handheld tattoo deviceif the force/pressure applied is not in the between the lower and upperoptimal force/pressure. When the operator is ready to perform apuncture, the operator can press the trigger element, as discussed instep 1314 of FIG. 17. If the contact force/pressure is in the optimalrange, the diagnostic system can report nominal operation and thehandheld tattoo device can confirm applying the parameter settings fromstep 1312. The handheld tattoo device can proceed immediately to step1315. In step 1315, the handheld tattoo device can perform a set ofpunctures (e.g., a puncture train). The depth, size, positioning, and/orpattern of the punctures can be specified in step 1312. In step 1312,the actuator(s) of the handheld tattoo device can be powered to puncturethe skin. A galvanic sensor and/or other sensors in the handheld tattoodevice can start measuring and recording signals related to theapplication of the set of punctures, the data can be buffered in thecontrol unit and, after the puncture train (e.g., set of punctures) ispartially or entirely completed, and the data can be formatted andtransferred to the control module. The data can be analyzed and/orinterpreted in step 1316. In some embodiments, the data is analyzed instep 1317. In step 1328, if i<n, the process returns to step 1311. Thesteps 1311 to 1316 are then repeated until a set or all the puncturinghas been completed.

Signal detection and/or interpretation can be used to analyze data andleads to the detection of the puncture events as described in connectionwith FIGS. 3A, 12A, and 12B, and the device setting for performing adesired dot (e.g., a circular dot, round dot, a dot with a desired size,color, color saturation, darkness, geometric characteristics, shape,etc.) at that location marker is computed by an algorithm describedherein. The device settings can be stored. The operator can then repeatthe operation from steps 1311 to 1317 for each dry marker until apredetermined number or all dry punctures have been performed.

Once performed, an optimal setting obtained in step 1317 for all the drymarkers of a specific dot size can be interpolated in step 1318 for allthe positions associated with wet dots of the same or similarconfiguration (e.g., size, shape, diameter, etc.). This is exemplifiedfor dot number 16 (dot 1105 in FIG. 18B), for which the settings can beinterpolated as a combination of the information collected from one ormore of dry punctures 2, 3, and 12. In one embodiment, thisinterpolation can be performed on different wet dot diameters than thedot diameters of the dry punctures by applying a setting conversion fromone type of needle to another type of needle used to obtain differentdot diameters. Wet dots are tattoo dots where a detectable amount of inkhas been applied to the subject. This can conclude the step 310 (FIG.3A) associated with the manual embodiment of the tattoo process.

In step 320 (FIG. 3A) associated with the manual embodiment of thetattoo process, the tattooing process can be performed. In the case ofthe manual embodiment of the tattoo process, wet dots can be applied.The operator can proceed to identify the marker associated with a wetpuncture in step 1321 of FIG. 17. These dots can be marked by a filledcircle in the stencil illustration in FIGS. 18A-18C. The diameter of thefilled circle can indicate the diameter of the needle used. In oneembodiment, small diameter circles are applied using a needle with 3 orless tips in a tight circular configuration and large diameter areapplied using a needle with 4 or more tips in a tight circularconfiguration. The position can either be identified through a numberapplied as part of the stencil and/or through a digital representationof the tattoo displayed on the computer or tablet that runs the controlsoftware's graphical user interface. Settings of the tattoo device areapplied in step 1322 (FIG. 17) by reading the interpolated parametersobtained in step 1318 (FIG. 17) and sending a command from the controlsoftware to the computing unit of the tattoo handheld unit. Once thesettings are applied in step 1322 (FIG. 17), the operator can positionthe contactor window or opening by centering the contactor with themarker in step 1323. In this step 1323, the operator also ensures thatthe needle (or needle assembly) is properly oriented perpendicular tothe skin surface (e.g., a longitudinal axis or line of action of theneedle assembly can be generally perpendicular to the surface of theskin) with or without using a level indicator. The operator can increaseor decrease the pressure applied by contactor to the skin by applyingmanual force. One of the operator's objectives can be to apply anoptimal level of force to the skin to ensure sufficient contact betweenthe contactor and the skin. The amount of force employed is measured bythe load cell (e.g., load cell 1008 of FIG. 16) and the value of theload can be digitally reported by the graphical interface of the controlsoftware. The optimal force allowed is between two force values, a lowerbound optimal force and a higher, upper bound optimal force. Theseforces can be between 0 g and 10,000 g. The control software can preventactuation of the handheld unit if the force applied is not within atarget range (e.g., between the lower and upper optimal forces).

When the operator is ready to perform a puncture, the operator triggersthe ink delivery system to inject ink in the needle well in step 1324 ofFIG. 17. For the initial wet dot, the needle well will be filled withink for the first time. For additional wet dots, a sufficient amount ofink can be present in the needle well and adding more ink may beperformed if necessary. When ready, the operator can press the triggerbutton in step 1325. If the contact force is in the optimal zone, thediagnostic system can report operation information and the handheld unitcan be configured based on the parameter settings from step 1322. Thehandheld unit then proceeds immediately to step 1326. In step 1326, thehandheld unit performs a set of punctures at the depth and numbersettings specified in step 1322. In this step, the handheld actuator canbe powered, which results in the needle (or needle assembly) puncturingthe skin. Data associated with the puncture event can be collected. Thiscollected data is analyzed and used to confirm appropriate applicationof the tattoo settings. The stored interpolation settings for subsequentdots may be adjusted if new settings are desired or needed to producedesired dots. In step 1338, if i<n, the process returns to step 1321.The steps 1321 to 1326 are then repeated until all the wet dots havebeen applied. This concludes step 320 (FIG. 3A) for the manual tattooingprocess embodiment.

In the manual embodiment of the tattoo process, step 308 of FIG. 3A canbe omitted as the handheld unit is not attached to the customer. Thetattoo area may be cleaned and dressed with a protection layer indressing tattoo step 309 (FIG. 3A). Additionally, a variety of aftercaretreatments may be provided. Finally, the surface of the tattooing devicemay be cleaned with a cleaning solution so as to be suitable forsubsequent use. Any disposable components may then be removed anddisposed of. In some embodiments, the handheld robotic units can beattached to the customer.

As used herein, the term “disposable” when applied to a system orcomponent (or combination of components), such as a needle, a tool, orstencil, is a broad term and generally means, without limitation, thatthe system or component in question is used a finite number of times andis then discarded. Some disposable components are used only once and arethen discarded. In other embodiments, the components and instruments arereusable and can be used any number of times. In some systems, all ofthe components can be disposable to prevent cross-contamination. In someother systems, components (e.g., all or some of the components) can bereusable.

Embodiments of the stencil deposition can be designed for manualapplication of the tattoo. One embodiment is displayed in FIGS. 18A-18C.The stencil can facilitate tattoo application by the manual operator andto provide a proxy reference of the final design to the client. In suchembodiments, the stencil can contain alignment markings 1101, 1102 and1103 (FIG. 18A) for proper operator alignment. Numbers and text 1104 tohelp the operator proceed in the correct order of dots. Filled circles1102 denote the locations designated for “wet” punctures. Unfilledcircles 1101 denote the locations for “dry” punctures to beadministered. Dry punctures are administered without ink, not part ofthe final tattoo, but used for gathering information about the skinpuncture properties prior to performing wet dots. One embodiment of thestencil uses various diameters for the filled and unfilled circles toindicate the correct needle type and arrangement for the tattoo dot sizeby varying the dot diameter. Any other markings may be present tofacilitate the operator application of the tattoo and customervalidation of design and placements. Stenciling procedures can includesteps discussed in connection with FIGS. 5A-6B, 11A-11D, and 18A-18C.

Digital Files, Computing Systems, and Controllers

FIG. 19 is a flow chart of a method 1208 for generating tattoo digitalfiles (or digital tattoos) that are optimized for automated tattooingand can be displayed as visual art or machine-read for the execution ofthe tattoo by the automated tattoo machine. These files are created byinterpreting graphical artworks or images.

In step 1210, a set of optional guidelines and directions may beprovided to the artist to facilitate the conversion of the artwork intoa digital tattoo. In some implementations, this step may be omitted.

In step 1211, art is received either physically or by other means indigital media formats which may be vector-based, raster-based, or acombination of both. In some embodiments, the artwork is received via awireless or wired connection. For example, the artwork can be receivedvia a local area network or wide area network.

In step 1212, the artwork can be converted into a digital image with astandardized format. If artwork is received physically, it can bescanned at a desired resolution (e.g., a high resolution using ascanner, or other imaging device) suitable for being converted into araster-based digital image. If artwork is received in digital mediaformat, any vector-based components of the artwork may be rasterized ata certain high resolution. The technique for converting the artwork to adigital image can be selected based on the desired processing time,resolution, and/or conversion accuracy.

In step 1213, the digital image can be preprocessed. For example, thedigital image can be preprocessed to adjust its brightness, contrast,light curves, dynamic range, color distribution, and/or enhance desiredgeometric features, such as edges. Separate preprocessing procedures maybe applied to different parts of the image, and manual touch-ups couldbe performed to achieve desired aesthetic.

In step 1215, a dot-based tattoo design can be generated by, forexample, using one or more conversion algorithms to convert the digitalimage into a collection of tattoo dots. This conversion may be performedin multiple stages, aimed to convert different aspects of the image,such as dots, lines and shades. The number of dots used to represent thetattoo design can be selected based on the resolution and capabilitiesof the tattoo apparatus.

In steps 1214 and 1215, different visual components of the artwork canbe detected and analyzed, such as individual dots, lines, shaded areasand edges of shaded areas. The analysis may be performed in multiplestages and parts of the image may be masked off at each stage to avoidduplicate detection of features. Isolated dots on the image whose sizeare similar to the tattoo dot or needle size may be identified asindividual tattoo dots and assigned a representative color or shade.Lines and edges of shaded or solid areas may be detected as lines with arepresentative color, shade and thickness, each of which may be varied.Line tracing techniques may be used to identify continuous lines oredges, and to construct a series of tattoo dots with varying spacing anddot size to represent the line with varying contrast, thickness orshade. The rest of the image, such as areas of varying color or shade,or continuous areas of a solid color may be covered with a collection oftattoo dots using space-filling methods, where the spatial densityand/or size of dots is varied to represent the variations of color orshade on the image. Space-filling may be performed based on anunderlying ordered grid, probabilistic dot placement, halftoning ordithering techniques. Computational stippling methods, such as oneutilizing weighted Voronoi cells, may be used to spatially rearrange thelocations of tattoo dots based on the gradients of color or shade on theimage. This operation may improve the visual representation of imagegradients by the spatial distribution of tattoo dots. The resultingcollection of tattoo dots constitutes a candidate tattoo design, and itis rendered on a screen in step 1216 for visual inspection by a humanoperator.

In step 1216, the tattoo design can be postprocessed. The candidatetattoo design may be compared with the original artwork on the screen tofacilitate the inspection. The tattoo design may also be digitallyoverlaid on pictures of body parts with different skin colors, to assessits aesthetic outcome. At this step, the operator may add, remove,and/or relocate dots manually to improve the aesthetic outcome of thetattoo design. Based on the outcome, the operator may also choose tomodify the image preprocessing settings of step 1213, and repeat steps1213-1216 to improve the design. As a result, the output of step 1216can be the final tattoo design, which visually represents the originalartwork received in step 1211.

In step 1217, metadata may be assigned to the tattoo dots to modifytheir puncture settings, such as the needle extension and number ofpunctures or ink delivery flowrate in the needle reservoir, around theirnominal values, in order to achieve a particular aesthetic aspect. Forexample, to better represent a light-shaded area of the artwork, thenumber of punctures or needle extension may be decreased for thecollection of tattoo dots in that area. Or, these settings could beincreased for very dark or color-saturated areas, to increase thedeposited ink per puncture and consequently reduce the time needed totattoo such areas.

In step 1218, a simplified design outline is generated from the art,which may be placed on the stencil to allow the customer to review thedesign's positioning on the skin before starting the tattooingoperation. A subset of the features detected in step 1214, such as themost distinct lines and edges, may be used to create the design outline.Positioning of the design may also be reviewed using augmented reality,wherein the final tattoo design (step 1216) is overlaid on a cameraimage or live video, based on the positioning and deformation of theapplied stencil on the image detected by machine vision. If augmentedreality is used, including the design outline on the stencil may not benecessary.

One or more of the steps 1218-1220 can be performed in parallel withsteps 1215-1217. For example, the lattice generation 1219 can beperformed concurrently with the step 1216. Each step can incorporatedata from other steps. For example, the tattoo metadata generation atstep 1217 can be based on the design reduction at step 1218. The orderand timing of the steps can be selected based on the tattoo file to begenerated.

In step 1219, a spatial arrangement of fiducial markers (lattice) isgenerated and placed on the stencil. The lattice of fiducial markers isemployed by machine vision to track the deformation of the skin anddetect its spatial coordinates (see FIGS. 5A-6B). This lattice isconstituted of fiducial markers of a certain diameter (e.g. 50-500 um)and spacing (e.g. 100-1000 um). This lattice may be spatially arrangedas a square grid, as shown in FIG. 5A, although other arrangements arepossible, such as a hexagonal grid, or unordered spatial distributions.Information-encoding variations (pattern) of these fiducial markers suchas location, size, omission, shape or color, (see FIG. 5A for a spatialvariation encoding example) may be used to create a pattern which allowsidentification of the local spatial coordinates by the machine vision.The pattern may be probabilistically (randomly) or deterministically(rule-based) generated.

In step 1220 of FIG. 19, the local-uniqueness and robustness of thepattern is checked. The requirement of local-uniqueness means: differentparts of the pattern, as exposed through the contactor window, must bedistinct for every potential position of the contactor. The robustnessof the pattern means that the local-uniqueness persists when parts ofthe pattern are concealed or when random noise is introduced to thepattern (these conditions may occur during tattooing operation or whenthe stencil is first applied to the skin). Deterministic orprobabilistic modifications may be made to the pattern, manually orautomatically, to improve the likelihood of either requirement beingsatisfied. Steps 1219-1220 can be repeated until a locally-unique androbust pattern is generated. In some embodiments, step 1219 and 1220 canbe performed in advance and the same lattice may be reused for variousstencils for tattoos of the same size.

In Step 1221, the digital tattoo data or file can be generated, whichcontains at least (i) the coordinates of the dots in the tattoo design(step 1216), and (ii) the coordinates of the fiducial markers (steps1219, 1220), (iii) a digital image of the stencil which (a) at leastcontains the fiducial markers and (b) may also contain the designoutline from step 1218. The tattoo file may also contain: (iii) adigital image of the original artwork (step 1212), (iv) artworkownership and licensing information, (v) the settings used in imagepreprocessing (step 1213), (vi) the metadata for tattoo dots (step1217), (vii) simplified outline of the art (step 1218) in vector form,(viii) a pre-computed data table to facilitate finding the tattoo dotsin a particular region. Each component in the digital tattoo file may bestored using the appropriate data structures for that component, suchas, a data table for the dot coordinates, a vector-based graphics formatfor the stencil, a raster-based graphics format for the artwork image,etc. As described in FIG. 3C the information in the tattoo file, such asthe components (i, ii, vi, vii), and the machine vision analysis (FIGS.5A-6B) are used in combination to create the machine instructions foreach tattoo dot, controlling (i) spatial positioning of the needle onskin, (ii) needle extension, (iii) number of punctures, (iv) amount ofink injected on the needle.

The accuracy, repeatability, capability, and/or resolution of therobotic application of the tattoo, as compared to traditional manualtattooing, may be characterized, for example by one or more of thefollowing. In one embodiment, the tattoo position (e.g., overall tattooposition, section of tattoo, etc.) relative to the absolute position onthe skin may be ±0.5 mm, ±1 mm, ±2 mm, ±3 mm, or ±4 mm in a skin planeas compared to a stencil positioning. In other embodiments, the overalltattoo position relative to the absolute position on the skin may be ≤±5mm, including but not limited to e.g. ±1 mm, ±2 mm, ±3 mm, ±4 mm, andall non-integer values e.g. ±0.6 mm, ±0.7 mm, ±1.2 mm, ±1.3 mm, etc. Therelative tattoo position can be selected based on the size, intricacy,resolution, or other features of the stencil, tattoo, or the like.

The optical detection (e.g., machine vision accuracy) may be ˜4 um perpixel. In other embodiments, the optical detection or machine visionaccuracy may be ≥˜4 um per pixel, or ≤˜4 um per pixel. In oneembodiment, the extracted position error of a fiducial marker may be≤±50 um in the skin plane, including but not limited to for example, ±40μm, ±35 μm, ±30 μm, ±20 μm, and all other non-integer values e.g. ±25.7μm, ±25.6 μm, ±25.5 μm, etc. The detection capabilities of the opticaldetection can be selected based on the characteristics of the tattoo andmay be better than detection via the naked eye.

The accuracy of the needle in the z plane may be ≤±100 μm from theprescribed needle elongation setting due to skin deformation, includingbut not limited to for example, ±90 μm, ±85 μm, ±80 μm, ±70 μm, and allother non-integer values e.g. ±65.7 μm, ±65.6 μm, ±65.5 μm, etc. In oneembodiment, the position accuracy of each tattoo dot compared to itsneighbors may be ≤±50 um, including but not limited to for example, ±40μm, ±35 μm, ±30 μm, ±20 μm, and all other non-integer values e.g. ±25.7μm, ±25.6 μm, ±25.5 μm, etc.

The expected resolution of tattooing in dots per inch (dpi) may be 72 to2540 dpi, but is variable based on design dot density. For example, theexpected resolution of tattooing may be, but is not limited to being,between 72 to 2540 dpi, or larger than 72 to 2540 dpi, e.g. between50-3000 dpi, etc. In one embodiment, the expected dot size may be, butis not limited to, between 100 um to 5000 um based on the needle size.In one embodiment, the expected tattooing speed may be ≤0.15 s per dot,for example including but not limited to 0.1 s per dot, 0.8 s per dot,0.5 s per dot, etc. In one embodiment, the expected time of completionof a 3.5×2 in, 15000 dots tattoo, including dry dots may be for example≤40 min.

FIG. 20 is a schematic block diagram illustrating subcomponents of acontroller 1400 in accordance with an embodiment of the disclosure. Thecontroller can be part of a control unit (e.g., controller 108 or 109 ofFIG. 1B) and/or can be incorporated into other tattoo device orcomponents disclosed herein. The controller 1400 can include a computingdevice 1402 having one or more of each of processors 1404, memory 1406,input/output devices 1408, and/or subsystems and other components 1410.The computing device 1402 can perform any of a wide variety of computingprocessing, storage, sensing, imaging, and/or other functions.Components of the computing device 1402 may be housed in a single unitor distributed over multiple, interconnected units (e.g., through acommunications network). The components of the computing device 1402 canaccordingly include local and/or remote memory storage devices and anyof a wide variety of computer-readable media.

As illustrated in FIG. 20, the processor 1404 can include a plurality offunctional modules 1412, such as software modules, for execution by theprocessor 1404. The various implementations of source code (i.e., in aconventional programming language) can be stored on a computer-readablestorage medium or can be embodied on a transmission medium in a carrierwave. The modules 1412 of the processor can include an input module1414, a database module 1416, a process module 1418, an output module1420, and, optionally, a display module 1422.

In operation, the input module 1414 accepts an operator input 1424 viathe one or more input devices, and communicates the accepted informationor selections to other components for further processing. The operatorinput 1424 can include, for example, stencil information, tattoo designinformation, subject preferences (e.g., preferences for tattoo, lengthof session, tattoo resolution, tattoo style, etc.), or the like. Theinformation can be displayed via the display 1422. The display 1422 canbe a touchscreen or other output device capable of displaying and/orreceiving input.

The database module 1416 organizes records, including internal andexternal variables, settings (e.g., machine settings, puncture settings,etc.), puncture parameters, scores (e.g., dot scores), subject datasets, experimental data, tattoo graphic data, stenciling data, artwork,tattoo designs, and operating records and other operator activities, andfacilitates storing and retrieving of these records to and from a datastorage device (e.g., memory 1406, an external database, etc.). Any typeof database organization can be utilized, including a flat file system,hierarchical database, relational database, distributed database, etc.

In the illustrated example, the process module 1418 can generate controlvariables based on sensor readings and/or image data 1426 from sensors,machine vision systems, and/or other data sources. The sensors caninclude, without limitations, impedance sensors, accelerometers,gyroscopes, contact sensors, pressure sensors, sensors configured tooutput signals associated with needle depth or position, galvanicsensors, or other suitable sensors.

The output module 1420 can communicate operator input to externalcomputing devices and control variables. The output module 1420 caninclude one or more communication elements, transmitters, receivers,antennas, ports (e.g., USB ports, LAN port(s), optical port(s), etc.),interfaces, etc. Example interfaces include USB port interfaces, a wiredLocal Area Network interface (e.g., Ethernet local area network (LAN)interface), a wireless network interface via a WiFi LAN access inaccordance with, for example, I.E.E.E. 802.11b/g/n wireless or wirelessnetwork communications standard. The display module 1422 can beconfigured to convert and transmit processing parameters, sensorreadings 1426, output signals 1428, via one or more connected displaydevices, such as a display screen, touchscreen, etc. the output signals1428 can be sent to one or more components to control or command thecomponents.

In various embodiments, the processor 1404 can be a standard centralprocessing unit or a secure processor. Secure processors can bespecial-purpose processors (e.g., reduced instruction set processor)that can withstand sophisticated attacks that attempt to extract data orprogramming logic. The secure processors may not have debugging pinsthat enable an external debugger to monitor the secure processor'sexecution or registers. In other embodiments, the system may employ asecure field programmable gate array, a smartcard, or other securedevices.

The memory 1406 can be standard memory, secure memory, or a combinationof both memory types. By employing a secure processor and/or securememory, the system can ensure that data and instructions are both highlysecure and sensitive operations such as decryption are shielded fromobservation. In various embodiments, the memory 1406 can be flashmemory, secure serial EEPROM, secure field programmable gate array, orsecure application-specific integrated circuit. The memory 1406 canstore instructions performing any of the methods disclosed herein,including, without limitation processing images, obtain informationabout our work and or tattoo designs, acquiring information, analyzingtarget sites, dot scoring, data collection, determining puncturesettings, digital stencil reference data, or the like. The memory 1406can include non-transitory computer-readable medium, memory component,etc. carrying instructions, which when executed, causes actions. Theactions can include steps disclosed herein.

The steps of the methods disclosed herein can employ one or more AItechniques. AI techniques can be used to develop computing systemscapable of simulating aspects of human intelligence, e.g., learning,reasoning, planning, problem solving, decision making, etc. AItechniques can include, but are not limited to, case-based reasoning,rule-based systems, artificial neural networks, decision trees, supportvector machines, regression analysis, Bayesian networks (e.g., naïveBayes classifiers), genetic algorithms, cellular automata, fuzzy logicsystems, multi-agent systems, swarm intelligence, data mining, machinelearning (e.g., supervised learning, unsupervised learning,reinforcement learning), and hybrid systems.

In some embodiments, image processing, detection (feature detection,fiduciary marker detection, reference feature detection), skin punctureproperty acquisition and analysis, skin color identification, positionanalysis, dot scoring, art conversion, artwork preprocessing, artworkanalysis, tattoo design generation, lattice generation, latticeverification, and other steps disclosed herein can use one or moretrained machine learning models. Various types of machine learningmodels, algorithms, and techniques are suitable for use with the presenttechnology. In some embodiments, the machine learning model is initiallytrained on a training data set, which is a set of examples used to fitthe parameters (e.g., weights of connections between “neurons” inartificial neural networks) of the model. For example, the training dataset can include any of the reference data stored in database 1416 (FIG.20), such as a plurality of reference puncture data sets or a selectedsubset thereof.

In some embodiments, the machine learning model (e.g., a neural networkor a naïve Bayes classifier) may be trained on the training data setusing a supervised learning method (e.g., gradient descent or stochasticgradient descent). The training dataset can include pairs of generated“input vectors” with the associated corresponding “answer vector”(commonly denoted as the target). The current model is run with thetraining data set and produces a result, which is then compared with thetarget, for each input vector in the training data set. Based on theresult of the comparison and the specific learning algorithm being used,the parameters of the model are adjusted. The model fitting can includeboth variable selection and parameter estimation. The fitted model canbe used to predict the responses for the observations in a second dataset called the validation data set. The validation data set can providean unbiased evaluation of a model fit on the training data set whiletuning the model parameters. Validation data sets can be used forregularization by early stopping, e.g., by stopping training when theerror on the validation data set increases, as this may be a sign ofoverfitting to the training data set. In some embodiments, the error ofthe validation data set error can fluctuate during training, such thatad-hoc rules may be used to decide when overfitting has truly begun.Finally, a test data set can be used to provide an unbiased evaluationof a final model fit on the training data set.

To generate a tattoo plan or protocol, a data set can be input into thetrained machine learning model(s). Additional data, such as the selectedsubset of reference patient data sets and/or similar patient data sets,and/or treatment data from the selected subset, can also be input intothe trained machine learning model(s). The trained machine learningmodel(s) can then calculate whether various candidate treatmentprocedures and/or medical device designs are likely to produce afavorable outcome for the patient. Based on these calculations, thetrained machine learning model(s) can select at least one treatment planfor the patient. In embodiments where multiple trained machine learningmodels are used, the models can be run sequentially or concurrently tocompare outcomes and can be periodically updated using training datasets. The module 1420 can use one or more of the machine learning modelsbased the model's predicted accuracy score.

The controller 1400 can include any processor, Programmable LogicController, Distributed Control System, secure processor, and the like.A secure processor can be implemented as an integrated circuit withaccess-controlled physical interfaces; tamper resistant containment;means of detecting and responding to physical tampering; secure storage;and shielded execution of computer-executable instructions. Some secureprocessors also provide cryptographic accelerator circuitry.

The input/output device 1408 can include, without limitation, atouchscreen, a keyboard, a mouse, a stylus, a push button, a switch, apotentiometer, a scanner, an audio component such as a microphone, orany other device suitable for accepting user input and can also includeone or more video monitors, a medium reader, an audio device such as aspeaker, any combination thereof, and any other device or devicessuitable for providing user feedback. For example, if an applicatormoves an undesirable amount during a tattoo session, the input/outputdevice 1408 can alert the subject and/or operator via an audible alarm.The input/output device 1408 can be a touch screen that functions asboth an input device and an output device.

The controller 1400 can detect events, such as adverse events. Theadverse events can include interruptions during the tattoo execution,such as (1) temporarily pausing and resuming the tattooing process, 2)stopping and later reinitiating the tattooing process, and 3) stoppingthe tattooing process due to an emergency or critical event and laterreinitiating the tattooing process. The operator can be notified via theinput/output devices 1408 of a detected event. Sensor readings 1426 canbe analyzed to automatically detect events based on sensor output. Thetattooing system 90 of FIG. 1B, shuttle 104 of FIG. 2, tattooingapparatus 400 of FIG. 8, tattoo device 900 of FIG. 14, and tattooingsystem 908 of FIG. 15 can detect events and may perform recoveryprocesses based on the detected event, as discussed below.

Example Tattooing Environment and Systems

FIG. 21 is a network diagram of a tattooing environment and system 1500(“system 1500”) in accordance with an embodiment of the disclosure. Thedescription of the tattoo system 60 of FIG. 1A applies equally to thesystem 1500 unless indicated otherwise. The system 1500 includes atattoo assistance system 1510 that can control tattoo apparatuses withdifferent configurations and functionality to allow operator(s) to applydifferent types of tattoos to multiple clients. A network 1560 canprovide communication between the various components of the system 1500via one or more wireless connections, wired connections, opticalconnections, or the like. In one embodiment, some or all of the tattooapparatuses are configured to be controlled by the tattoo assistancesystem 1510 and can be located at a single tattoo studio or separatetattoo studios.

FIG. 21 illustrates the system 1500 including a tattooing apparatus 1520with features and functionality described in connection with FIGS. 1A,1B, and 2, a tattoo apparatus 1530 with features and functionalitydescribed in connection with FIG. 8, a handheld tattoo apparatus 1532with features and functionality described in connection with FIGS.14-16, and tattoo apparatuses 1540, 1550. The number, configuration, andfunctionality of the tattoo apparatuses and components can be selectedbased on the number of subjects to be tattooed at the same time,characteristics of the tattoos to be applied, desired length of tattoosession, preferences of the operator or subject, or the like. In someembodiments, different needle heads and/or needles can be utilized bythe tattoo apparatuses 1520, 1530, 1532, 1540, 1550. Different types ofend effectors, needle assemblies, tattoo needles, fluidic systems,controllers, or the like can be selected based on desired tattoocapabilities.

An ink delivery system can be used to provide ink to a single tattooapparatus or to multiple tattoo apparatuses. Referring again to FIG. 21,the tattoo apparatus 1540 includes a robotic tattoo apparatus ormulti-axis robotic arm 1570 (e.g., 3-axis robotic arm, 4-axis roboticarm, 5-axis robotic arm, 6-axis robotic arm, etc.), an end effector inthe form of a needle head assembly 1580, and an ink delivery system1582. The ink delivery system 1582 includes containers 1583 (e.g., inkbottles, ink cartridges, etc.), a fluidic system 1584 (including hoses,valves, pumps, etc.), and other fluidic components for fluidicallycoupling the ink delivery system 1582 and the needle head assembly 1580.The tattoo apparatus 1540 can include a machine vision device or system1600 (“machine vision system 1600”) with multiple image capture devices1602 (e.g., spaced apart digital cameras) for imaging-based automaticinspection and analysis. In other embodiments, the devices 1602 areLiDAR sensors. The needle head assembly 1580 can also include one ormore sensors for providing data to the tattoo assistance system 1510 viacommunication channel 1620.

The tattoo apparatus 1550 can include an end effector in the form of aneedle head assembly 1621 with an integrated machine vision system,sensor(s) (e.g., contact sensors, optical sensors, mechanical sensors,chemical sensors, light detectors, galvanic sensors, pressure sensors,etc.), or other features disclosed herein. The integrated machine visionsystem can be protected by the housing of the needle head assembly 1621.The tattoo assistance system 1510 can concurrently support and providefunctionality to different machine vision systems for each tattooingapparatus. This allows the tattoo studio to utilize various types ofmachine vision systems and apparatuses. The vision and/or sensor datacan be used to control one or more tattoo steps.

An ink delivery system 1592 can be in fluid communication with thetattoo apparatuses 1520, 1530 and can include one or more pumps, lines,fittings, and other features (e.g. fluidic systems 1594, 1596) forindependently delivering ink to the tattoo apparatuses 1520, 1530. Insome embodiments, a single ink delivery system can deliver ink to all ofrobotic tattooing machines.

The handheld tattoo apparatus 1532 having ink delivery system 1535including a fluid container or a cartridge (illustrated as a syringecompatible with a housing of the tattoo apparatus 1532), fluid line, andother fluidic components. Ink cartridges are discussed in connectionwith FIGS. 14 and 16. The handheld tattoo apparatus 1532 can be used tomanually apply tattoos, touchup tattoos or dots applied robotically,and/or apply a portion of the tattoo while another portion of the tattoois applied robotically. In some implementations, a single tattoo designcan be applied by sequentially applying portions that tattoo usingrobotic tattoo apparatus (e.g., robotic tattoo apparatus 1520, 1530,1570, or 1550) while the portions of the tattoo are applied concurrentlyor sequentially by the handheld apparatus 1532. The handheld tattooapparatus 1532 (or tattoo apparatuses of FIG. 21) can include one ormore output modules to communicate with and provide data to thetattooing environment and system 1510. The tattooing environment andsystem 1510 can adaptively control the tattoo apparatus applying thetattoo based on the received data from another tattoo apparatus. Thisallows for coordination between tattoo apparatus to produce a desiredtattoo. In non-tattoo implementations, the robotic tattoo apparatus1520, 1530, 1570, or 1550 and handheld apparatus 1532 can be used toperform aesthetic treatments.

The tattooing apparatus 1532 can include ink container 1535. Forexample, the ink container 1535 may include an ink delivery systemdescribed in connection with FIG. 16. The tattooing apparatus 1532 andother tattooing apparatuses disclosed herein can include a pump orrefilling system for replenishing ink by, for example, replacing orrefilling the ink container 1535. The tattooing apparatus 1532 can berefilled when the ink is at a low level or at a rate commensurate to thenumber of punctures performed. The tattooing apparatus 1532 can refillor replace multiple ink containers to avoid downtime.

With continued reference to FIG. 21, the tattoo assistance system 1510can include one or more controllers, displays 1640, input devices 1650,etc. and can include features or functionality disclosed herein. In someembodiments, the tattooing environment and system 1500 has multipletattoo assistance systems 1510 each incorporated into the tattooingapparatuses. The tattoo assistance system 1510 and controllers disclosedherein can be programmed to identify events, including low ink supplies,machine vision alerts, temporary pauses, stop events, emergency orcritical events, adverse events, or the like. The tattoo assistancesystem 1510 and controllers disclosed herein and can also be programmedto monitor and provide information such as ink levels, tattoo apparatusdata (e.g., status, maintenance history, operational history, settings,etc.), client data (e.g., preferences, profiles, payment, orderhistories, etc.), tattoo design data, or the like.

In operation, the tattoo assistance system 1510 can generate a tattooprotocol to apply a tattoo and can communicate with the tattoo apparatusto be used. The tattoo assistance system 1510 can store one or morecontrol maps, command programs, instruction sets, and other data forcontrolling the tattoo apparatus to be used. For example, the controlmap for the tattoo apparatus 1540 can include angles for controllingeach of the joints of the robotic arm. In 6-axis robotic armembodiments, the control map can include angles for each of the 6 jointsto position the needle device 1580. Additionally or alternatively, thecontrol map can include target pose data for positioning an end effectora desired location. For example, the control map can include translationdata, rotation data, or other data with reference to one or morereference frames. Based on a target location of the end effector toapply a dot, the tattoo assistance system 1510 can determine thetranslation and rotation data and commands for moving the end effectorto the target location.

The tattoo assistance system 1510 can implement one or more programs forenforcement regarding authorization, authentication, and/orconfiguration of the tattoo apparatuses. In some embodiments, thetechnology disclosed herein can be incorporated into a commerciallyavailable robotic system. The tattoo assistance system 1510 cancommunicate with the robotic system and obtain control data forcontrolling the robotic system. The control data can include, withoutlimitation, number of degrees of freedom, geometric parameters ofcomponents of the robotic apparatus, force settings, range of motiondata, pose data, tolerance data, or the like. The tattoo assistancesystem 1510 can generate one or more machine settings (e.g., settingsfor selected poses), control maps, command programs, instruction sets,kinematic model data, and other data (e.g., position matrices, Jacobianmatrices, transformation matrixes, joint vectors, rotational vectors,translational vectors, etc.) based on the received control data.

In non-tattoo setting, the robotic tattoo apparatus 1520, 1530, 1570, or1550 and handheld apparatus 1532 can a apply botulinum toxin,anti-wrinkle agents, denervating agents, anti-acne agents, collagen,medicants, or the like. The system can optically analyze a site andidentify wrinkles (using a trained computer vision system similar tothat described above). Targeted wrinkles located along the subject face(e.g., along the forehead, surrounding the eyes, etc.) or any otherlocation. The apparatus can determine one or more puncture sites basedon characteristics (e.g., size, depth, location, etc.) of the wrinkles.The apparatus can inject one or more anti-wrinkle agents at puncturesites to reduce or limit the appearance of the targeted wrinkles. Thesystem can perform both medical and aesthetic procedures. In anotherimplementations, each robotic tattoo apparatus 1520, 1530, 1570, or 1550and handheld apparatus 1532 can apply ink, dyes, or other substances toarticles, such as purses, belts, and other articles of manufacturedisclosed herein.

FIG. 22 is a block diagram of an embodiment of a tattooing method 1700in accordance with an embodiment of the disclosure. In general, thetattooing method 1700 can be used to robotically apply a tattoo based onvisual artwork. The tattooing protocol may (i) use a set of internal andexternal data as inputs 1710 related to tattoo execution, (ii) use a setof algorithms or methods 1720 which control tattoo execution based onthese inputs, and/or (iii) generate machine instructions 1742 which uponexecution by an embodiment of the tattoo system or machine 1750. Atblock 1760, a tattoo visually replicating the original artwork or designis applied. Details of the method 1700 are detailed below.

The inputs 1710 may include, for example, the digital tattoo file (e.g.,output of method 1208 in FIG. 19), skin puncture property table (e.g.,table 316 of FIG. 3C), and/or other skin puncture data collected bysensors, or data or output of machine vision-based methods (e.g.,process 357 of FIG. 7). In some embodiments, all of the inputs are usedto determine an execution control program. In other embodiments, themethod can include selecting subsets of input or data. The skin punctureproperty table can include puncture property data for a relatively largearea. Systems disclosed herein can identify and select a subset of thepuncture property data corresponding to the site to which the tattoowill be applied. The selected subset of data can then be used togenerate execution control data.

The control algorithms 1720 may include, for example, (i) methods andsystems used to calculate the coordinates of tattoo dots on the skin1721, such as those described in connection with FIGS. 5A,5B,6,7, and/or(ii) methods to identify or modify puncture settings 1722 (e.g., needleextension, number of punctures, etc.) to achieve desired visualcharacteristics for each tattoo dot, such as methods described inrelation to FIGS. 12A,12B, and 13. The tattoo protocol may be executedby a controller, computer, or processor 1741, which uses the controlalgorithms 1720 as well as other operational inputs 1730 to generatemachine instructions 1742 to perform tattooing. Example to operationalinputs 1730 include: inputs to control the rate of ink injection,control lighting for the machine vision system, triggers to capturecamera images, turn suction system on or off, any requests to stop orinterrupt the operation from the user or from safety sensors (e.g.,motion sensors, vibration sensors, accelerometers, contact sensors,gyroscopes, etc.), etc. The generated machine instructions 1742 are thentransmitted to the tattoo system or machine 1750.

The instructions, upon execution by the tattoo machine 1750, causes theactuation of the components to perform the operations on the skin toapply the tattoo by, for example, moving the tattoo head and/or needle,applying tattoo dots, injecting of ink, cleaning the skin, flattening ormoving the skin, interrupting operation, recovering from error events,etc. Error detection and error correction techniques may be used in thetransmission of machine instructions, such as repetition codes, paritybits, checksum, cyclic redundancy check (CRC), Hamming codes, etc., toensure the correct and intended instructions are executed on the skin.

FIG. 23A shows an example of a visual art received from an artist. Asdescribed in FIG. 19, a digital tattoo image or file can be generatedbased on the original artwork which contains at least a collection oftattoo dots to be placed on or formed in the skin, in addition to theother information for robotically executing the tattoo. An example of agenerated collection of tattoo dots is shown in FIG. 23B, which mayresult from processing the artwork in FIG. 23A. The dots in the tattoofile may have uniform or varying dot-to-dot spacing, dot size, color,and/or ink saturation, in order to create an accurate visualrepresentation of the original artwork. Referring to step 326 in FIG.3C, in order to tattoo each dot in the window, machine actuationsettings are determined, which includes of at least (i) the targetcoordinates of the dot on the skin within the window (see coordinates inFIG. 6B, calculated by method 357 of FIG. 7) and/or (ii) the puncturesettings to achieve the desired dot characteristics (e.g., needleextension and number of punctures, read from the dot parameter table 316of FIG. 3C, or based on the method 808 in FIG. 13). The machineactuation settings for the dot can be encoded as machine instructions toactuate the machine, for example: T,DOT,x,y,z,n may encode a machinetask (T) to perform n number of punctures for a dot (DOT) at windowcoordinates x,y with a base needle extension z. The machine instructionsmay be encoded into a series of bytes or bits, appended with anappropriate error detection code, and then transmitted to the machinevia a secure wireless channel, serial or parallel data cable, etc.

The machine can decode the received machine instructions, and performsan error check based on the method used. If no error is detected, themachine executes the actuation of the gantries and the needle asprescribed to apply the tattoo dot. If a communication error (forexample a bit flip, bit omission or interference in the transmittedinstruction) is detected, an error message is sent back to the processorto interrupt the operation or re-transmit the machine instructions. Inaddition to actuating the needle as explained above, other operationalcommands in the form of machine instructions may be transmitted to themachine, for example, instructions to (i) move the gantries (e.g.,gantry 105, gantry 107, etc.), or the robotic arm, which houses thetattoo head, (ii) capture images from the machine vision camera (e.g.,machine vision camera 131, 430, etc.), (iii) actuate the ink pump (e.g.,pump of fluidic system 1584), and/or (iv) turn the suction system 150on/off, etc. The executable instructions can be executed to coordinateoperation between components of the systems and apparatuses disclosedherein.

The robotic tattooing systems can automatically form tattoo dots atlevels of consistency, accuracy, and/or speed which cannot be achievedby human tattoo artists. Visual outcome of tattoo dots may be quantifiedby one or more of the following: (i) dot location, (ii) dot size, (iii)color intensity, (iv) total ink content of the dot, and/or (v) dot 2Dand/or 3D geometry (e.g. including 2D imaged geometry and depthinformation). Needle actuation may be controlled by the puncturesettings, including (i) number of punctures and/or (ii) needleextension. The needle actuation can be along a line of action that isgenerally perpendicular to the surface of the skin or at another desiredorientation.

FIGS. 24A-24C illustrate an example of the effect of compensation of thedeformation of a portion of the skin when tattooing. The tattooingsystem can compensate for changes of the skin to produce a desiredtattoo design. The changes can be deformation of the skin caused by, forexample, repositioning of the subject's body part, shear forces appliedto the skin, pressure applied to the skin, etc. The system can identifythe skin changes and select one or more algorithms to compensate for theidentified changes. The deformation of the skin can be analyzed bycomparing the position of fiducials taken in an undeformed, relaxedstate to the current position of the fiducials.

FIG. 24A shows the reference tattoo design 1800 in a targetconfiguration when skin is relaxed. The system can apply a tattoo thatmatches the reference tattoo design 1800 by modifying the tattooingprotocol based on changes in position, deformation, and/or state of thebody part. This allows tattooing to proceed when the body part moves,skin is stretched, or other unforeseen events occur. The tattooingsystems disclosed herein can continuously or periodically monitor theskin and/or components of tattoo systems to modify the tattooing in realtime. Compensation routines can be performed during a single tattoosession or between tattooing sessions and are discussed in connectionwith FIGS. 5A, 5B, 6A, 6B, and 7. Example outcomes of tattoo operationswith and without skin deformation compensation are discussed inconnection with FIGS. 24B and 24C.

FIG. 24B shows the stretched edge of the portion of the skin 1820 in adeformed state, whereas FIG. 24C shows the same area of skin 1820 in anundeformed or natural, at rest, state. The subject's body part may bedeformed due to, for example, positioning the body part on a supportsurface, such as a tattooing pad. This may cause the tissue of the bodypart to be in a deformed or unnatural state. If skin deformation is notcompensated for, the machine vision may track the position of the skinin the state in which the tattoo is performed.

FIG. 24B shows the compensated tattoo design 1810 b on the deformed skin1820 and an uncompensated tattoo design 1810 a, which matches thereference design 1800 of FIG. 24A. The uncompensated tattoo design 1810a is applied without deformation compensation on the deformed skin 1820and is an accurate rendition of the reference tattoo design 1800 of FIG.24A. However, when the skin is returned in its undeformed state, shownin FIG. 24C, the uncompensated tattoo design 1810 a is completelydeformed and not an accurate representation of the reference design 1800in FIG. 24A. The compensated design 1810 b is applied to the skin asshown in FIG. 24B. The design 1810 b appears deformed compared to thereference design of FIG. 24A. However, when the skin is returned to itsrelaxed, natural or undeformed state, in FIG. 24C, the design of thetattoo 1810 b appears as an accurate rendition of the reference designof FIG. 24A. The systems and controllers disclosed herein can alterdesigns to match the deformation of skin such that the design has thedesired configuration when the skin is relaxed or undeformed.

FIG. 25 shows a high-magnification photograph 3011 of a series of fourtattoo dots robotically applied to human skin. In order to testconsistency of the resulting tattoo dots, the same puncture settingswere used for applying each dot. Bottom panel 3010 in FIG. 25 shows datacollected from a galvanic sensor during the execution of each tattoodot. The robotic system performed five punctures for each dot in lessthan 60 ms at a generally uniform rate of needle oscillation, detailedbelow.

The puncture settings included five punctures per dot and a needleextension of 350 μm beyond the exposed skin surface. Each rise in thesignal readings corresponds to the needle coming into contact with theskin. The number of times the signal rises for each dot is equal to thenumber of punctures (5) prescribed to the machine, demonstrating theaccuracy of the system in executing punctures. For example, dot #1 wasformed by puncturing a location 5 times with the same needle. The tip ofthe needle was moved to a maximum depth 350 μm. As shown, the tattooingsystem was capable of consistently producing dots with a target size,for example, dot sizes with a ±10% deviation (±22 μm deviation) around adot size of 225 μm. The dots are generally circular with well-definedperipheries. The puncture events were performed in less than 60 ms at agenerally uniform rate of oscillation. The needle oscillation rate canbe varied to, for example, compensate for changes at the puncture site,adjust for volumes of ink delivered for each puncture event, etc. Thenumber of punctures for each location, needle extension (e.g., extensionfrom a defined location), volume of ink delivery per puncture event, orother puncture settings can be selected based on the dots to be formed.

The tattooing system can achieve a positional accuracy with a targetpositional range. For example, the tattoo system applied the illustrateddots with a positional accuracy of 10-50 μm in the placement of tattoodots on skin. To blend adjacent dots, for example, the positionalaccuracy could be increased. The positional accuracy can be increased toproduce high-resolution micro tattoos. The positional accuracy can beselected based on the design of the tattoo. The puncture settings of thetattooing system can be inputted and/or modified by the user. In someembodiments, puncture settings are generated by the tattooing system. Acombination of puncture settings from the user and generated puncturesettings can be used. A user can review and modify the settings tocustomize the tattooing protocol based on user expertise. In otherembodiments, the tattooing system generates a set of puncture settingsthat can be modified by a user after a checker confirms that themodifications conform to one or more criteria (e.g., tattoo qualitycriteria, safety criteria, pain management criteria, etc.). The puncturesettings can be optimized puncture parameters determined by thetattooing system. The tattooing systems disclosed herein can beprogrammed to reduce or limit errors, such as location errors. Thesystem can identify and correct for (i) error in robotic or gantry-basedactuation (˜0-10 μm) and (ii) measurement noise in machine vision system(˜0-50 μm) which compensates for in-plane deformation of the skin. Thisallows dots to be accurately positioned throughout a portion of or anentire tattoo procedure. In comparison, positional accuracy of a humanhand holding a tool tip may be on the order of 100-250 μm due to naturaltremor, wander and jerk motions, according to experiments published inhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3459596/.

FIG. 26 shows a series of tattoo dots produced with varying puncturesettings to control the visual outcome of tattoo dots. The top image3110 shows the effect of varying the needle extension setting (left toright, extensions of 250 μm, 350 μm, 450 μm beyond the skin surface).This shows how the needle extension influences the resulting dotcharacteristics (e.g., shape, size, etc.). The bottom image 3111 showstattoo dots applied with 5 and 10 punctures per dot, at the same needleextension setting of 350 μm beyond the upper skin surface. The number ofpunctures influences the resulting characteristics of the dots. Theresulting dot size, color intensity and total ink content are calculatedbased on image analysis techniques. The experiments show that all threevisual properties can be efficiently controlled by the puncturesettings.

Tattoo dots can form the building blocks of a tattoo design. The tattoodots may not fall on a rectangular grid, which allows optimized spatialresolution. High resolution tattoos can be created by tattooing systems,remote servers, etc. For example, resolutions as high as 5 dots permillimeter may be achieved, based on a mean tattoo dot size of 200 μm. Atattoo with dimensions 10 cm by 5 cm may contain approximately 125,000tattoo dots, where the visual appearance of each dot may be controlledby varying the puncture settings as shown in FIGS. 25 and 26. In someembodiments, the dot pitch can be 5 μm, 10 μm, 25 μm, 50 μm, 100 μm, 250μm, 500 μm, 1 mm, 2 mm, or another pitch and the mean dot size can be100 μm, 200 μm, 300 μm, 400 μm, 500 μm or another mean dot sizes.

Cleaning Operations

Systems disclosed herein can perform one or more cleaning operationsbefore, during, and/or after tattooing. Before tattooing, cleaning cansanitize the tattooing site. During tattooing, cleaning can removeexcess fluids, such as ink, bodily fluids, lubricant, etc. Cleaningallows for a clear observation of the skin by machine vision systems.The lubricant can have multiple roles. First, it acts as a lubricant forthe contactor and needle to provide the proper interaction (e.g.,sliding along the skin, gliding against the skin, etc.) with the skin.Needle lubrication can facilitate puncture, and contactor lubricationcan inhibit excessive friction between the skin and the contactor.Second, the lubricant can act as a stain barrier. For example, when inkis applied on skin, the ink may stain the skin if no lubricant isapplied to create a barrier. Without lubricant, the cleaning action isdifficult and ink stain may remain after the removal of the excess ink.Through application of a lubricant, such as a hydrophobic silicone-basedcompound, a barrier is created over the skin. The ink drop may not crossthe lubricant barrier, thereby preventing staining of the skin. Theexcess ink may be removed without resulting in staining or mitigatingstaining of the skin.

The contactor can also facilitate cleaning. The contactor can be firmlyin contact with the skin during tattooing such that any excess fluidsare kept within the contactor window. If no contactor is used, the inkand other fluid may runoff the tattoo area, stain other portions of theskin and potentially carry pathogen outside the tattoo area. Thecontactor can help reduce the extent of the cleaning as well asprotecting against contamination. Suction systems can be used to removeexcess fluid. The suction head may be integrated to the contactor windowedge for edge suction and/or the suction may include a nozzle that runsacross the tattoo window in the vicinity of the needle cartridge. Theshape of the nozzle can be selected to provide suction in allconditions. For example, a badly positioned nozzle may be too far fromthe skin to remove small drops of excessive liquids. In another example,the nozzle may be positioned too close to the skin and may seal againstthe skin, removing the protective lubricant protective layer. The nozzledistance and angle of attack are selected to remove all excess liquidwithout removing a layer of lubricant and without sealing against theskin. A nozzle angle of attack may be inclined (e.g., a longitudinalaxis of the nozzle may be inclined from the normal to the skin) suchthat the nozzle does not form a seal (e.g., an airtight seal,fluid-tight seal, etc.) with the skin, allowing the nozzle to be asclose to the skin as desired. The excess fluid suctioned off the skin bythe suction system may be collected in a collection container anddiscarded. The suction line may also be disposable or sterilizable.

Some amount of skin staining may be acceptable if the observation by themachine vision system can be performed effectively as described herein.The cleaning action may be repeated if the machine vision system cannotperform its tasks effectively. For example, residual ink may occult thefield of view of the machine vision. The machine vision step used inposition may be unable to perform the position and deformation analysisand may trigger additional cleaning. The additional cleaning may removethe occlusion due to excess fluid.

Movement and Position Detection

The tattooing systems disclosed herein can be operated based, at leastin part, on detection of movement, positional information, needledetection, or the like. The tattooing process may be paused in responseto detecting movement of the skin being tattooed. This movement may bedetected by a single sensor or multiple sensor embodiment. Machinevision can be used to visualize the skin and may be used to detectchanges in skin position. If the skin changes position, the visible setor pattern of fiducials (e.g. pattern 342 b in FIG. 6A) may be alteredand detected using machine vision. A global approach may also be used bycomparing consecutive images during the tattoo process. If a changeoccurs, the two consecutive images may be different which may result inrapid detection of any potential movement. The image variation may thenbe evaluated to determine whether a movement actually occurred. Forexample, some pixel color and density value from one picture may becompared to the color and density value of a consecutive picture. Thiscan be done very rapidly compared to the speed of tattooing. If thedifference is large enough, the full evaluation of the portion of theskin position by pattern analysis (for example method 357 of FIG. 7),which is a slower process compared to tattooing, can be performed. Therapid analysis process may be hindered by the appearance of excess fluidand camera movement, resulting in a slower confirmation process.

Other sensor(s) may be used to simplify the process. For example, asecondary optical sensor may be integrated to the contactor to observechanges from image to image of a portion of the skin that is not beingtattooed. The detection by the optical sensor may be used to trigger afull observation of the position of the portion of the skin beingtattooed by the machine vision system. In some embodiments, the opticalsensor includes a light source (e.g., a light-emitting diode (LED)) andone or more light detectors (e.g., an array of photodiodes). The lightdetector(s) can output images or other data for determining movement ofthe skin.

Non-optical sensors and systems may be used. The non-optical sensors caninclude one or more accelerometers, gyroscopes, vibration sensors, etc.,that may be used to detect skin movement. For example, when the skinmoves, this movement may be picked up as a vibration which triggers apause and an evaluation of the position of the skin. Additionally oralternatively, the dielectric property of the skin may be used toevaluate movement. In the case of movement, the electrical path betweenthe needle electrode and the measurement electrode is changed, resultingin a variation in impedance. This variation may be used to detectmovement of the portion of the skin being tattooed. The coordinate ofthe tattoo dot would likewise be updated to account for the change ofposition of the portion of the skin being tattooed. A controller (e.g.,controller 1400 of FIG. 20) can be programmed to compensate for thechange of skin position.

Tattooing Processes and Events

A temporary pause may be initiated during the tattoo process by eitherthe operator, the client or the machine itself. In case of a client oroperator interruption, a command or tactile switch may be available. Inthe case of a machine triggered interruption, a warning message may bedisplayed (e.g., via the input/output 1408 or display 1422 of FIG. 20)to inform the operator that a pause was triggered as well as an error orwarning message. In normal operation triggering, the aforementionedtemporary pause may cause the tattoo machine to pause the tattooingprocess. Setting granularity can include, but is not limited to, pausingbetween dots or between the executions of all the dots within a tattoowindow. When the tattoo process is ready to resume the tattoo operatoror client may resume the tattooing process without a requirement forrecalibration or reinitiating of the tattoo machine. The operator mayinstead decide to trigger a stop if the tattoo process cannot be resumedor at the request of the client.

A stop of the tattoo process may be triggered by the operator, usingeither a switch or a command line. The tattoo progress information isdumped from the core to a restart file to assist with the eventualresuming of the tattooing process, and the actuators are put in a safeposition which would allow the client to disengage from the machinesafely. In the case, alignment may be disrupted and a recalibration ofthe tattoo machine may be required. To resume tattooing after a stop,the client can reengage with the machine and the tattoo area can becentered in the tattoo frame. Recalibration is achieved by initiallyperforming a scan (e.g., a partial or comprehensive scan) of the tattooarea before moving in the vicinity of the last completed dot area andrecommencing the tattooing process. If the tattoo stencil is notsufficiently preserved for the performance of machine vision, a newstencil may be applied and machine vision can assess the location of thenext tattoo dot by scanning the completed portion of the tattoo. Becauseit may be difficult for an operator to exactly align a stencil with apartially completed tattoo, the new applied stencil lattice is freefloating. This means that the position of the reference tattoo design isnot initially fixed within the lattice. The machine vision can be usedto scan the tattoo area completely, in particular the already completedarea of the tattoo. This allows using a digital image correlation orother image analysis method to identify the exact position of thepartially completed tattoo in the newly applied lattice. The position ofthe reference design within the digital reference lattice is thencalculated from this digital image correlation and the tattoo processcan be resumed where the partial tattoo was initially stopped. The sameor similar strategy may be used when the expected tattoo is larger thanthe tattoo frame, in which case the stop function is triggered toposition the client's skin such that the non-tattooed part of the tattoois now centered in the tattoo frame while still some of the completedpartial tattoo is also visible to provide sufficient machine visioninformation for position referencing. This stop and shift strategy isrepeated until the tattoo is fully completed.

The systems can provide optimal tattoo frame placements for a tattoodesign. The frame placements can be displayed (e.g., inserted, overlaid,etc.) on a reference image of the body part with stenciling, a referencestenciling image (e.g., an image of the applied stenciling), etc. Forexample, a display (e.g., display of controller 108 of FIG. 1B, display1640 of FIG. 21, display 1422 of FIG. 20, etc.) can display the frameplacements with respect to the reference image.

In some embodiments, systems provide positioning features for locatingthe non-tattooed skin with respect to the tattoo frame or anothercomponent. In some embodiments, the system includes one or moreprojection lighting devices (e.g., LED lighting devices, laser devices,etc.) configured to project one or more images (e.g., arrows, tattooboundary markers, targets for centering in a window of the tattoo frame,etc.) on the skin or frame. After the skin is positioned with respect tothe frame, the system can analyze and confirm proper placement. Ifadjustments are needed, additional positioning information can beprovided to the user.

In some embodiments, stenciling can include positioning information forsequentially positioning the body part with respect to the tattoo frame.For example, the positioning information can be used to align the bodypart with one or more features of the frame by, for example, centering anon-tattooed part of the body part. The system can analyze completedportions of the tattoo and can, if needed, instruct the user to move thebody part to enable tattooing to be resumed based on the fiducial and/orapplied dots. The positioning information can be reference frames,sizing features, targets, locators surrounding fiducials, etc. In someembodiments, the system generates positioning features based on analysisof fiducials, applied portions of tattoos, and/or other referencefeatures. Additionally or alternatively, one or more positioningfeatures can be integrated into the tattoo frame and be activatabledirection indicators, such as light sources (e.g., arrow-shaped lightsources).

When generating a tattooing protocol for large tattoos, the systemsdisclosed herein can generate a positioning protocol to be provided to auser. To avoid long periods of uncomfortable tattooing, the system candetermine sequences of tattooing for pain management. For example,tattoo sections can be assigned a pain score and a protocol can begenerated based on one or more criteria, such as maximum length ofsubstantially continuous tattooing with a threshold pain score,anticipated pain based on tattooing area (e.g., sensitive areas have ahigh pain or discomfort score, etc.).

In the event of an emergency or critical event (as determined by theclient and/or the tattoo operator), a command or tactile emergencyswitch may be available for that purpose. In normal operation,triggering the emergency switch will cut power to the actuators(directly and indirectly) and dumps the tattoo progress information fromthe core to a restart file to assist with the eventual resuming of thetattooing process. The passive safety of the actuator may allow theclient to remove themselves from the tattoo machine when the actuator isunpowered. The resume function of the tattoo in case of an emergencystop is similar to the one for a stop.

The restart data and/or files for the emergency stop and standard stopmay be transferred to the cloud or to a detachable storage media and maybe used in another machine altogether. This may allow completion onanother machine, for example, in case of critical failure of the machineor if the client wishes to complete their tattoo at anotherlocation/store/shop. The restart data and/or file may contain theoriginal tattoo file, the ID of the machine that performed the work,diagnostics of the machine at the time of tattoo, ID of tattooed ortested dot in the design and ID of remaining dots, the dot parametertable for the tattoo, the raw data files collected by the sensors andmachine vision systems and all other data generated in the originalsession. Other information in the restart file may include identifyingthe tattoo session and client information as well as other informationinput by the operator.

The tattoo systems may also encompass recovery methods in the case ofmachine malfunction. Based on the gravity of the machine malfunction, awarning or a pause or a stop or an emergency stop may be initiated bythe operator, automatically by the machine or by the client.

External malfunction may include loss of reliable power, such as duringa power outage. One embodiment of the invention includes anuninterruptible power supply (UPS) which allows providing power in caseof outage, at least long enough to complete the ongoing tattoo. In casethe tattoo is not finished within the predicted battery life of the UPS,the operator or machine itself may trigger the stop function of theequipment.

The automated tattoo machine includes automated and manual diagnosticfunction that evaluates if the device is operating nominally. As part ofthis disclosure, we present some of the diagnostic function for criticalsystem. This is not construed as exhaustive and it shall be assumed thateach subsystem has its own operational diagnostic function to verifynominal operation.

One potential source of malfunction is a disposable malfunction, inparticular a needle malfunction, an electrode malfunction or an inkdelivery malfunction.

Needle malfunction may be identified by a change of the galvanicresponse of the needle when in contact with the skin, a change in theperceived dot quality by the machine vision system, by the operator orthe client observation or response or by ink delivery to the skinfailure as observed by the machine vision. In case of a needlemalfunction, the machine may trigger an error message and pause themachine and/or the operator may trigger a pause. The operator may decideto trigger a stop if a replacement of the needle is warranted. Theclient is allowed to disengage from the machine while the operatordiagnose and address the needle malfunction by issuing a needlereplacement. The tattoo process may resume as specified by the stoppingprocess. If the needle cartridge was replaced, dry dots may need to beresumed from the start in order to account from the variability ofneedle sharpness and length which may affect puncture settings. Notethat if dry dots are done a second time, their position with respect tothe tattoo may be shifted slightly in order to avoid puncturing the skinat the location of previous punctures as this may shift themeasurements.

Electrode malfunctions are identified by the addition of a testelectrode or internal circuitry which purpose is to verify resistance ofeach electrode connection. In case of electrode failure, the process ofthe tattoo may be paused to replace the electrodes. The or internalcircuitry electrode may also be replaced. Electrode contact resistancemay be tested at the beginning and throughout the tattoo process toverify operation. A stop may or may not be triggered by the operatordepending on whether or not dry punctures need to be reevaluated.

Ink delivery malfunction may be detected when the ink delivery is tooclose to the capacity of the reservoir, if no ink is observed to exitthe needle tip or if the dot on the skin seems to be executed with aninappropriate amount of ink. A pause or stop may be triggered to refillthe reservoir, exchange the ink delivery line or replace the needlecartridge. In case of no disruption to the tattoo process, a pause maybe sufficient. In case the needle cartridge is replaced, the processspecific to needle replacement may be executed.

Detected actuation failure may trigger a pause (for transient failuresuch as motor overheating), stop, or an emergency stop (for a power ormechanical failure) in order to protect the client. The operator maydecide to resume the process at a later time and trigger a maintenanceflag for the machine.

In general consideration, any diagnostic error from the machine maytrigger a pause, a stop or an emergency stop, which may be addressed bythe operator during the tattoo session or by a subsequent maintenance.Corrective action (positive or negative) may be taken in response to anyerrors, malfunctions, failures, or other adverse events (e.g., excessskin deformation, machine vision errors, etc.), such as, but not limitedto, those described throughout this application.

The robotic systems can use a dot database. The number of punctures fora specific ink dot can be referred to in the dot database. This is thenumber of times the needle will touch and puncture the skin at the samelocation for the purpose of transferring ink. This number of puncturesaffects the final size and color intensity of the ink dot. The tattoodevice can pilot the number of punctures performed at a certain positionto achieve various tattoo dot diameter and for varying the colorintensity to achieve various area coverage in the design and for colortone and color gradient with the same ink. The number of punctures atthe same location can be varied from 1 to 100 punctures which the systemalgorithm attributes to different tone, gradient and dot size. Puncturenumber at a location can be selected to vary gradient, tone and/or dotsize. The robotic system can include an ink quality monitor configuredto monitor the ink quality based on, for example, ink viscosity, opticalcharacteristics of ink (e.g., color intensity, tone, etc.), or the like.The robotic system can determine the number of punctures at a locationbased, at least in part, on ink characteristics, such as viscosity,optical characteristics, retention in skin, etc. For example, the numberof punctures can be increased or decreased for high color intensity inkor low color intensity ink, respectively.

Tracking and Positioning Technologies

The positioning algorithm disclosed in this patent in relation to FIGS.5,6,7 is distinguished from previously available tracking-based globalpositioning algorithms. Tracking-based global positioning methods andalgorithms can be used to calculate the relative positioning of asurface of interest with respect to the camera, or vice versa. Thesemethods can operate by (i) collecting a series of images (e.g., imagesfrom one or more cameras) of the surface of interest at different timesor corresponding to different relative positioning of the camera(s) andthe surface, (ii) identification of unique features on the surface ofinterest, either pre-determined or selected during operation, (iii)tracking the positions of the features on each image over time, and (iv)calculating the relative positioning of the surface of interest withrespect to the camera(s), based on the geometric relationships betweenthe positions of tracked features on the camera image. Tracking-basedglobal positioning methods for controlling robotic arms to manipulateobjects in 2D or 3D space can be used. Tracking-based global positioningmethods can pose certain limitations in their application for automatictattooing. First, the features should be sufficiently distinct from eachother in appearance, to be uniquely identified and tracked acrossdifferent images. This may limit the number of distinct tracking pointswhich may be placed on the surface (e.g. “surface” being the skinsurface in a tattooing application). Second, the precision of theidentified locations of the features is typically limited the order of amillimeter, which may prevent use of the technology for high-precisiontattooing applications. For example, based on a 10 mega-pixel camerataking images from a distance of 1-2 meters (typical operating space ofa robotic arm). These two aspects would limit the resolution, accuracyand precision of tattoos robotically applied on a body part based on atracking-based global positioning method.

In contrast, a pattern-detection based machine vision method, forexample, as described in relation to FIG. 7, may be used to overcome theabove limitations and identify the position of a tattooing head, as wellas the local deformation of the skin, with a much higher precision(e.g., detected skin position within 10-100 um of actual skin position).The pattern-based method described in FIG. 7 can include approaching theskin surface with a robotic system to capture close-up, high-resolutioncamera images of a pattern of fiducial markers on the skin. The systemcan identify the position of the camera relative to the skin bysearching for the detected pattern within the entire pattern known priorto actuation. In this method, the machine vision camera (e.g., machinevision camera 131 in FIG. 2, machine vision camera 430 in FIG. 8, etc.)is attached to (and moves with) the tattoo shuttle (e.g., tattoo shuttle104 in FIG. 2, or tattoo shuttle 404 in FIG. 8), which may be positionedon the body by a gantry system or a robotic arm. The images beinganalyzed can be close-up, high-resolution images of the surface (asubset of the area being tattooed), which dramatically increases theresolution, accuracy and precision.

Patterns can be analyzed. For example, referring to step 367 of themethod in FIG. 7, the position of the tattoo head on the body isidentified by matching the partial pattern in the image, to the entirepattern, which is performed by a search algorithm. In some methods, theimages do not need to contain the entire tattoo area, which wouldrequire a minimum distance to the skin and thus would decrease theresolution of the machine vision-based image analysis and consequentlywould decrease the spatial accuracy of the tattoo. In some methods, thepattern of fiducials may be either naturally-occurring or syntheticallycreated and applied on the skin, for example, using a stencil (See step530 in FIG. 9). For example, FIG. 6A shows an image captured by amachine vision camera, which reveals a portion of a pattern. The patternin this example is produced by omission of stencil dots, while thevisible dots inform the local deformation and rotation of the skin (step366 in FIG. 7). After analysis of fiducials and the pattern they form(See FIG. 6B), the position of the tattoo head is identified (e.g., step367 of FIG. 7) by searching for the detected portion of the patternwithin the entire pattern (See FIG. 5A). The machine vision methodsdisclosed can eliminates the need to track individually-unique featuresfrom a minimum viewing distance, which are at the limiting aspects oftracking-based global positioning methods discussed herein.

In some tattooing methods, tracking techniques are used concurrently orsequentially. Global positioning can be used to analyze and track theposition of body part, stenciling, and other identifiable features fordeveloping a tattooing routine. Machine vision methods can then be usedto track individual features at the tattoo site while applying ink tothe site. In some methods, multiple tracking techniques are usedsimultaneously for tracking redundancy.

Multi-Stage Tattooing

A multi-stage tattooing process may be performed to achievemulti-spatial tattooing. Each stage can apply portion(s) of the tattoowith specific characteristics. A high-precision stage, for example, canbe performed for high spatial precision tattooing (e.g., achieving aspatial accuracy in placement of tattoo dots within 10-50 um of theirtargeted locations) on generally curved body parts. A low-precisionstage, for example, can be performed for rapid tattooing of a relativelylarge area. An example two-stage process can include (1) globalpositioning of a tattoo head on the body part, coming in stable contactwith the skin, and (2) local positioning and actuation of a tattooneedle with high precision. This method can substantially increase theprecision of tattoo execution by moving the burden of positionalaccuracy from the global positioning stage to the local positioningstage. The first global positioning stage corresponds to the grossspatial positioning of the tattooing head on the body part, which may becurved. The actuation of the tattooing head may be achieved by, forexample, a multi-axis robotic arm (e.g., 6-axis robotic arm), a 2 axisor 3 axis gantry system, or a combination of both where the tattooinghead is attached to a gantry system through additional actuators, thusallowing rotational and translational movement of the tattooing head.The spatial control of the first stage (i.e., the global positioningstage) to position the tattoo head on a desired part of the body may beperformed by a combination of technologies, such as LIDAR-based 3Dsurface reconstruction, machine vision systems based on trackingreference features on the skin, etc. The precision of a globalpositioning stage alone may be in the order of a millimeter or more(e.g., 0.5 mm, 1 mm, 2 mm, 3 mm, 4 mm, 5 mm, etc.) due to (i)vibration-based and actuation-based limitations of a robotic arm, and(ii) optical limitations of a machine vision camera placed at adistance. Such systems may be unable to achieve high spatial accuracyand resolution in tattoo execution (e.g., tattoo dots, tattoo lines orother design features applied within 1-50 μm, 10-100 μm, 20-150 μm, etc.of their targeted locations).

The multi-stage process enables the desired accuracy, precision, and/orresolution in the second stage (i.e., the local positioning stage),which starts after the tattooing head comes into contact with the skin.The presence of one or more machine vision devices attached to thetattooing head can provide target spatial precision, including highspatial precision. For example, the machine vision technology describedin connection to FIGS. 5A, 5B, 6A, 6B, and 7 may be used in a second orhigh-precision stage, to (i) identify the precise position of thetattooing head on the skin, (ii) identify the local deformation androtation of the skin under the tattooing head with high precision, (iii)map the coordinates of dots on the tattoo design onto theircorresponding coordinates on the skin, and (iv) accordingly position thetattoo needle on the skin surface to execute the tattoo dots with highspatial accuracy and precision. The desirable accuracy of the two-stageor multi-stage process results from performing the machine visionanalysis and the subsequent control of the tattoo needle, only after thetattooing head comes into contact with the skin. For example, suchcontact may be achieved by the contactors or contactor systems disclosedherein.

Pain Management

The system disclosed herein can be used to manage pain by using one ormore pain inhibitors. The pain inhibitors can include one or moreanalgesic elements configured to cool tissue an effective amount toinhibit, limit, or substantially prevent pain. Analgesic elements canbe, for example, Peltier devices, thermoelectric cooling elements,cryo-elements capable of applying cryogenic or cooled fluids to controlthe temperature of tissue being tattooed via conduction, convection, orcombinations thereof. For example, tissue can be cooled to or below ananalgesic temperature such that the temperature of the tissue remainscooled during piercing. The analgesic effect can minimize, limit, orsubstantially prevent pain felt by the client during the injectionprocess or portion thereof. The tattoo or inking head or anothercomponent can include or carry analgesic elements configured to producean analgesic effect without thermally damaging the tissue.

In some procedures, tissue can be cooled to a temperature equal to orlower than an analgesic temperature at which nerve tissue is at leastpartially numbed to block temperature-induced pain signals from beingperceived by the brain. Additionally, the system can control thetemperature of the targeted tissue to prevent or control tissue freezingto prevent unwanted freezing pain and/or injury. Without being bound bytheory, cooling of the epidermal and dermal tissue can create aconduction block in epidermal, dermal and sub-dermal sensory nervefibers innervating these tissues, thereby providing an analgesic effect.In addition to the blocking or reduction of nerve conduction sensorynerve fibers for prevention and/or reduction of acute somatic painperception, local cold exposure may also reduce post-puncture swelling,inflammation, and bleeding, through vasoconstriction, and thereby reducepain and fear associated with the tattooing process. In someembodiments, cooling can be used post injection to inhibit, limit, orsubstantially prevent unwanted side effects (e.g., swelling,inflammation, pain, etc.).

The cooling can create temporary or reversible conduction blocks insensory nerve fibers innervating tissue, thereby providing the analgesiceffect. In one procedure, a target area or site can be rapidly numbed inless than about 5 seconds, 10 seconds, 1 minute, 5 minutes, 10 minutes,20 minutes, 60 minutes, 90 minutes, or other desired cooling period. Theanalgesic elements or cooled fluid (e.g., blown air, flowing liquid,etc.) can be at a temperature within a range of about −20° Celsius toabout 5° Celsius, about −15° Celsius to about 5° Celsius, or about −5°Celsius to about 2° Celsius, or other suitable temperature ranges forachieving desired analgesic effect. In some embodiments, cooling ratesof the skin surface or targeted tissue can be equal to or greater thanabout 0.01° C./minute, 0.1° C./minute, 1° C./minute, 5° C./minute, orother desired cooling rates selected based on, for example, clientcomfort. The target area tissue can be at a temperature less than 0°Celsius, 5° Celsius, 10° Celsius, 15° Celsius, or other suitabletemperature when punctured. The target temperature can be selected basedon the number of injection sites to be tattooed within a period of timeand desired analgesic effect. A controller or tattooing module can beprogrammed to cause the system to cool tissue from normal temperature toa cooled temperature to anesthetize the bulk tissue at the target areaor site. For example, the target tissue can be cooled to a temperatureequal to or lower than about 0° C., 1° C., 2° C., 3° C., 4° C., 5° C.,6° C., 7° C., 8° C., 9° C., 10° C. or 15° C. The skin can be monitoredusing one or more temperature sensors, optical sensors, or freeze detectsensors to avoid and/or counteract adverse cooling events, such astissue freezing.

CONCLUSION

The construction and arrangement of the elements of the systems andmethods as shown in the embodiments are illustrative only. Although onlya few embodiments of the present disclosure have been described indetail, those skilled in the art who review this disclosure will readilyappreciate that many modifications are possible (e.g., variations insizes, dimensions, structures, shapes and proportions of the variouselements, values of parameters, mounting arrangements, use of materials,colors, orientations, etc.) without materially departing from the novelteachings and advantages of the subject matter recited. For example,elements shown as integrally formed may be constructed of multiple partsor elements. It should be noted that the elements and/or assemblies ofthe apparatus may be constructed from any of a wide variety of materialsthat provide sufficient strength or durability to, for example,repeatedly apply tattoos. Any embodiment or design described herein isnot necessarily to be construed as preferred or advantageous over otherembodiments or designs. Accordingly, all such modifications are intendedto be included within the scope of the present inventions. The order orsequence of any process or method steps, including the steps discussedin connection with the algorithms discussed herein may be varied orre-sequenced according to alternative embodiments. Other substitutions,modifications, changes, and omissions may be made in the design,operating conditions, and arrangement of the embodiments withoutdeparting from scope of the present disclosure or from the spirit of theappended claims. For example, the techniques disclosed herein can beused to tattoo different articles, including articles made of naturalmaterials, synthetic materials, or the like.

The present disclosure contemplates systems and methods which may beimplemented or controlled by one or more controllers to perform theactions as described in the disclosure. For example, in someembodiments, the controller, whether part of a tattooing apparatus or aseparate controller, may be configured to process the measured data fromthe sensors, perform the recording, appending, or storing of the data(e.g., puncture data, ink data, needle data, skin data, etc.) and/or anycalculated values within the different tables or maps described, performall described and any similarly suitable algorithms, and controloperation of any disclosed parts or components in a manner necessary orappropriate for proper function, operation, and/or performance of anydisclosed systems or methods. For example, the controllers (e.g.,controller 108, controller 109, etc.) can store data and calculatevalues based on the stored data.

The controllers can include machine-readable media and one or moreprocessors, Programmable Logic Controllers, Distributed Control Systems,secure processors, memory, and the like. Secure storage may also beimplemented as a secure flash memory, secure serial EEPROM, secure fieldprogrammable gate array, or secure application-specific integratedcircuit. Processors can be standard central processing units or secureprocessors. Secure processors can be special-purpose processors (e.g.,reduced instruction set processors) that can withstand sophisticatedattacks that attempt to extract data or programming logic. A secureprocessor may not have debugging pins that enable an external debuggerto monitor the secure processor's execution or registers. In otherembodiments, the system may employ a secure field programmable gatearray, a smartcard, or other secure devices. Other types of computingdevices can also be used.

Memory can include memory, such as standard memory, secure memory, or acombination of both memory types. By employing a secure processor and/orsecure memory, the system can ensure that both data and instructions arehighly secure. Memory can be incorporated into the other components ofthe controller system and can store computer-executable orprocessor-executable instructions, including routines executed by aprogrammable computing device. In some embodiments, the memory can storeprograms for preset configurations. Stored programs (e.g., tattooingprograms, calibration programs, graphic mapping programs, etc.) can bemodified by a subject, operator, or tattoo artist to provideflexibility. Tattooing programs can be configured for tattooing animals,articles, goods, or the like. For example, some tattooing programs canbe for tattooing animals (e.g., living humans or farm animals) and othertattooing programs can be for tattooing articles (e.g., purses,footwear, clothing, automobile seats, etc.).

Controllers can be in communication with the components of the tattooingapparatus via, for example, a direct wired connection, a wirelessconnection, or a network connection. The controller 108 of FIG. 1B, forexample, can be a handheld electronic device, such as a tablet, smartphone, or the like, and it can include digital electronic circuitry,firmware, hardware, memory, a computer storage medium, a computerprogram, processor(s) (including programmed processors), or the like. Inother embodiments, the controller 108 is a computer (e.g., a tower, adesktop, or a laptop) connected to the apparatus 100 via a wired orwireless connection. The controller 108, or the controller 109, can beused to modify stencils, as discussed in connection with FIG. 3A. Thecontroller 109 (illustrated schematically in FIG. 1B) can include inputelement in the form of buttons or a touch screen for individuallycontrolling the apparatus. In one embodiment, controllers 108 and 109can include a user interface for inputting, modifying, and/orcontrolling any system component or process step described in thisdisclosure.

The present disclosure contemplates methods, systems, and programproducts on any machine-readable media for accomplishing variousoperations. The embodiments of the present disclosure may be implementedusing existing computer processors, or by a special purpose computerprocessor for an appropriate system, incorporated for this or anotherpurpose, or by a hardwired system. Embodiments within the scope of thepresent disclosure include program products comprising machine-readablemedia for carrying or having machine-executable instructions or datastructures stored thereon. Such machine-readable media can be anyavailable media that can be accessed by a general purpose or specialpurpose computer or other machine with a processor. By way of example,such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROMor other optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to carry or storedesired program code in the form of machine-executable instructions ordata structures, and which can be accessed by a general purpose orspecial purpose computer or other machine with a processor. Wheninformation is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combinationof hardwired or wireless) to a machine, the machine properly views theconnection as a machine-readable medium. Thus, any such connection isproperly termed a machine-readable medium. Combinations of the above arealso included within the scope of machine-readable media.Machine-executable instructions include, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing machines to perform a certain function orgroup of functions.

The headings provided herein are for convenience only and are notintended to limit or interpret the scope or meaning of the technology.Although the figures may show a specific order of method steps, theorder of the steps may differ from what is depicted. Also, two or moresteps may be performed concurrently or with partial concurrence. Allsuch variations are within the scope of the disclosure. Likewise,software implementations could be accomplished with standard programmingtechniques with rule based logic and other logic to accomplish thevarious measuring steps, calculating steps, storing steps, calibratingsteps, and any other steps for proper coordination and operation of thesystems and methods described above. Aspects of the described technologycan be modified, if necessary, to employ the systems, functions, andconcepts of the various references described above to provide yetfurther embodiments. For purposes of this disclosure, the terms customerand subject are interchangeable. Tattoos can be applied to animals(e.g., skin of mammals, including humans, pigs, cattle, farm animals,etc.), articles, natural materials (e.g., leather), synthetic materials,or other tattooable items. For example, tattoos can be applied toleather goods (e.g., belts, wallets, backpacks, etc.) using the systems,tattoo apparatus, and methods disclosed herein. In one embodiment, thetattooing system 90 of FIG. 1B can be configured to tattoo a leatherboot, bag, or other article, for example. While the above descriptiondetails certain embodiments and describes the best mode contemplated, nomatter how detailed, various changes can be made. Implementation detailsmay vary considerably, while still being encompassed by the technologydisclosed herein. The various aspects and embodiments disclosed hereinare for purposes of illustration and are not intended to be limiting,with the true scope and spirit being indicated by the following claims.

What is claimed is:
 1. A tattoo apparatus for applying a tattoo,comprising: a tattoo device having at least one needle; a target siteanalyzer configured to obtain information about a subject's skin; and acontroller that executes programming which causes the tattoo apparatusto: receive the obtained information from the target site analyzer,generate a protocol for applying at least a portion of a tattoo based onthe received information and a tattoo design, wherein the protocolincludes puncture setting data obtained by detecting a skin punctureevent and identifying a target layer of the subject's skin based on thedetected skin puncture event, wherein the puncture setting data is usedfor controlling piercing depth to position a tip of the at least oneneedle in the identified target layer to apply ink, and/or puncturelocation data for selecting individual puncture sites along the skin toinject ink, wherein the puncture location data is obtained from analysisof skin position and skin deformation, and robotically apply, via thetattoo device, the portion of the tattoo by injecting ink in the targetlayer at the selected individual puncture sites according to theprotocol.
 2. The tattoo apparatus of claim 1, wherein the target siteanalyzer includes: a machine vision device configured to obtain one ormore images of a portion of the subject's skin, and/or at least onesensor configured to measure at least one characteristic of the portionof the subject's skin.
 3. The tattoo apparatus of claim 1, whereingenerating the protocol includes: determining a skin position and a skindeformation based on the obtained information, and generating commandsfor controlling piercing depth, number of punctures, and puncturelocations based at least in part on at least one of the skin position,the skin deformation, or a characteristic of the portion of skin.
 4. Thetattoo apparatus of claim 1, further comprising: identifying one or moreskin changes capable of impacting a visual appearance of the tattoo tobe applied based on the received information, and generating at least aportion of the protocol based on the identification of the one or moreskin changes.
 5. The tattoo apparatus of claim 1, wherein the protocolis for applying a respective dot at a respective puncture site along thesubject's skin based on analysis of the subject's skin at otherlocations.
 6. The tattoo apparatus of claim 1, wherein the tattooapparatus is configured to detect an adverse event and to take acorrective action in response to the detection of the adverse event. 7.The tattoo apparatus of claim 8, wherein the adverse event relates to atleast one of: an interruption to tattoo execution, a needle malfunction,an electrode malfunction, an ink delivery malfunction, a controllermalfunction, an actuation failure, a software malfunction, or adiagnostic error.
 8. A tattoo apparatus, comprising: a tattoo devicehaving a needle configured to apply pigment to a subject's skin; amachine vision device configured to obtain one or more images of aportion of the subject's skin; and a controller that executesprogramming which causes the tattoo apparatus to: receive one or more ofthe images from the machine vision device, analyze the received one ormore images to identify reference features, analyze deformation betweena relaxed configuration of the skin and a deformed configuration of theskin based on the identified reference features, determine puncturelocations along the skin to compensate for the deformation configurationin relation to a tattoo design, and robotically apply, via the tattoodevice, dots to the skin at the determined puncture locations.
 9. Thetattoo apparatus in claim 8, wherein the controller executes programmingto further cause the tattoo apparatus to: identify individual dotsapplied to the skin using the machine vision device to determineposition information of the identified dots in relation to the tattoodesign; and robotically apply, via the tattoo device, additional dotsbased on the determined position information.
 10. The tattoo apparatusof claim 8, further comprising at least one sensor that measures atleast one characteristic of the portion of the subject's skin, whereinthe controller executes the programming to further cause the tattooapparatus to: calculate a skin position and/or a skin deformation basedon the received one or more images, and control a piercing depth basedat least in part on at least one of skin position, the skin deformation,or the at least one characteristic of the portion of the subject's skin.11. The tattoo apparatus of claim 8, wherein the controller executes theprogramming to cause the tattoo apparatus to: identify one or morechanges associated with the skin capable of impacting a visualappearance of the tattoo to be applied, and compensate for the one ormore changes associated with the skin to robotically apply the at leasta portion of the tattoo.
 12. The tattoo apparatus of claim 11, whereinthe one or more changes include skin stretch, skin displacement, andskin layer thickness changes, wherein the identified reference featuresinclude stenciling applied to the skin.
 13. The tattoo apparatus ofclaim 8, wherein the controller executes the programming to reduce oneor more differences between a selected tattoo design and the tattooapplied to the skin by determining the puncture sites matching theselected tattoo design.
 14. The tattoo apparatus of claim 8, wherein thecontroller executes the programming to (a) identify one or more of skinstretch, skin movement, an appearance of one or more dots applied to theskin by the tattoo device, and/or one or more changes to a stencilcorresponding to a selected tattoo design, and (b) compensate byselecting puncture locations for applying ink based on theidentification such that the applied tattoo appears to the naked eyesubstantially identical to the tattoo design.
 15. The tattoo apparatusof claim 8, wherein the controller includes: one or more processors; andmemory with instructions stored thereon, which when executed by the oneor more processors, cause the tattoo apparatus to: apply one or morereference features to the skin, analyze the identified referencefeatures in the received one or more of the images to evaluate one ormore characteristics of the skin to determine one or more changes in theskin, and compensate for the one or more changes in the skin todetermine puncture sites for applying the pigment.
 16. The tattooapparatus of claim 8, further comprising at least one sensor configuredto measure information about a puncture operation on the portion ofskin.
 17. The tattoo apparatus of claim 8, wherein the controllerexecuting the programming further causes the tattoo apparatus to:identify one or more changes associated with the skin and/or stenciling,and compensate for the one or more identified changes to roboticallyapply the at least a portion of the tattoo by one or more of thefollowing: (a) determining at least one relationship between the one ormore changes to the tattoo design and selecting one or more puncturelocations based on the determined at least one relationship, (b)selecting one or more puncture locations using characteristics of theskin with the one or more changes, and/or (c) generating a virtualtattoo design with corresponding changes such that the virtual tattoodesign simulates how the tattoo would appear on the skin with the one ormore changes.
 18. The tattoo apparatus of claim 8, wherein thecontroller is programmed to determine a stenciling protocol based on aselected tattoo; apply one or more locators to the skin according to thestenciling protocol; and robotically apply, via the tattoo device, thetattoo using the one or more locators such that the tattoo applied tothe skin is within a threshold deviation of the selected tattoo.
 19. Thetattoo apparatus of claim 8, wherein the controller is programmed to (a)receive one or more of additional images from the machine vision device,(b) analyze the received one or more of additional images of step (a) todetermine one or more of the puncture locations, (c) roboticallypuncture the skin at the determined one or more puncture locations toapply using the tattoo device at least a portion of the tattooassociated with a region of skin captured in the received one or more ofthe images of step (b), and (d) repeat steps (a)-(c) until the tattoo isapplied.
 20. The tattoo apparatus of claim 19, wherein the repeating ofsteps (a)-(c) are performed in relation to sequential regions of atarget area of skin to which the tattoo is to be applied.
 21. The tattooapparatus of claim 19, wherein the controller is programmed to cause thetattoo apparatus to apply a majority of dots forming the tattoo at thepuncture locations determined based on one or more reference features,wherein the one or more reference features are applied to or naturallypresent on the skin.
 22. The tattoo apparatus of claim 21, wherein theone or more reference features are a pattern of temporary fiducialmarkers applied to the skin.